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Ultra-cold  atom  system  provides  novel  technology  to  simulate  traditional  solid 
state  physics,  including  boson  and  fermion  particles.  Due  to  the  flexibility  of  tuning  pa¬ 
rameters,  people  can  further  understand  basic  physics  behind  strongly  correlated  effects, 
especially  mechanism  of  unconventional  density  wave  and  superfluidity.  Using  weak  cou¬ 
pling  renormalization  group  method,  we  propose  and  study  several  models  which  can  be 
realized  in  experiments.  Ranging  from  most  well-known  extended  Hubbard  model,  to  spin- 
polarized  Fermi  Hubbard  model  and  multi-flavor  Fermi-Fermi  mixture,  we  establish  solid 
theories  to  explain  the  origin  of  new  states  of  matter,  and  the  experimental  techniques  to 
exploit  them.  The  effects  of  lattice  structure  and  particle  density  in  fermionic  system  play 
important  roles  for  determining  phase  diagram  in  low  energy  scale.  Due  to  the  presence  of 
lattice,  these  systems  have  insulator  phase  at  half-filled.  The  density  imbalance,  however, 
prevents  formation  of  conventional  density  wave  state,  and  the  screening  interaction  dra¬ 
matically  affects  other  species’s  behavior.  The  interplay  between  screening  interaction  and 
square  lattice  is  key  ingredient  of  unconventional  density  wave  and  superfluidity.  Our  study 
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shows  that  the  unconventional  density  wave  and  superfluidity  can  be  realized  and  detected 
in  ultra-cold  atom  experiments. 
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Chapter  1 


Introduction 


Theoretical  Condensed  Matter  Physics  (CMP)  strives  to  understand  the  proper¬ 
ties  and  behaviors  of  matter.  One  of  most  intriguing  phenomena  of  interacting  quantum 
many-body  systems  in  CMP  is  the  emergence  of  new  states  of  matter  and  competition  be¬ 
tween  different  types  of  order.  From  a  methodological  view,  a  physical  system  would  be 
very  desirable  for  developing,  understanding  and  verifying  a  theory  if  it  can  be  described 
with  as  few  tunable  parameters  as  possible.  The  spectacular  progress  of  experiments  has 
provided  a  new  arena  for  studying  correlated  quantum  many-body  phenomena,  where  the 
microscopic  Hamiltonian  can  be  realized,  and  in  addition,  the  parameters  are  possibly 
controllable  by  tuning  external  fields  [1].  Various  lattice  structure,  including  cubic,  honey¬ 
comb,  and  kagome,  can  also  be  realized  by  different  sets  of  counter  propagating  lasers  [2]. 
Comparing  to  conventional  solid  state  materials,  these  techniques  can  provide  a  clean  and 
controllable  environment  which  is  closed  to  theoretical  model.  One  might  hope  that  this 
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reliable  technique  can  shed  the  light  on  discovering  insightful  physics  of  the  emergence  of 
new  states  of  matter. 

The  problem  of  understanding  physical  mechanism  in  strongly  correlated  fermionic 
system,  like  metal-insulator  transition,  origins  of  anti-ferromagnetism  and  superconductiv¬ 
ity,  remains  an  important  challenge.  Despite  the  large  effort  in  the  literature,  many  issues 
still  remain  controversial,  mainly  because  of  the  absence  of  unbiased  and  reliable  theoretical 
methods  for  handling  the  strong  correlation  problem.  In  the  weak  coupling  case,  at  least  in 
principle  it  is  tractable  by  perturbative  techniques.  The  main  idea  of  renormalization  group 
(RG)  is  integrating  out  the  high  energy  modes,  and  obtaining  a  low  energy  scale  physics  in 
CMP.  The  famous  review  article  [3]  provides  a  basic  idea  and  procedure  on  this  treatment. 
It  is  clear  that  further  investigation  is  interesting  and  necessary  for  complicated  solid  state 
materials  and  novel  fermionic  system  in  ultra-cold  gas  experiment. 

In  order  to  understand  the  origin  of  unconventional  density  wave  and  superfluid, 
we  propose  and  study  several  models  which  can  be  realized  in  ultra-cold  atom  experiment. 
Considering  the  lattice  structure  and  particle  density  in  correlated  fermions  system,  such  as 
extended  Hubbard  model,  spin-polarized  Hubbard  model,  and  quantum  degenerate  Fermi- 
Fermi  mixture,  we  conclude  that  effective  long  range  interaction  and  density  imbalanced 
are  the  key  to  formation  of  unconventional  density  wave  and  superfluid  in  low  energy  scale. 
The  experimental  setup  and  detection  of  different  phases  are  discussed  in  each  cases. 

The  present  dissertation  is  divided  as  follows;  Chapter  2  reviews  several  important 
concepts  and  techniques  used  in  the  thesis.  Chapters  3  to  6  then  present  my  main  work. 
Chapter  7  gives  summary  and  Appendix  A  provides  more  details  discussed  in  Chapter  2. 
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More  specifically,  in  chapter  2,  we  briefly  summarize  the  procedure  of  RG  method 
on  weakly  interacting  fermion  system.  By  integrating  out  the  thin  energy  shell  systemati¬ 
cally  on  one-loop  RG  equation  for  four  point  vertex,  we  arrive  a  series  of  effective  Hamil¬ 
tonians  at  different  RG  scale  as  we  proceed  this  systematic  process.  According  to  the 
proper  symmetry  of  the  problem  and  background  knowledge,  we  can  define  different  insta¬ 
bility  channels,  for  instance,  density  wave  or  superconductivity,  then  we  can  analyze  these 
channels  and  determine  any  instabilities  of  the  system. 

In  chapter  3,  we  study  the  phase  diagram  of  the  extended  Hubbard  model  on  a  two- 
dimensional  square  lattice,  including  on-site  (U)  and  nearest-neighbor  (V)  interactions,  at 
weak  couplings  [4] .  Despite  there  were  several  studies  on  Hubbard  model  by  using  functional 
renormalization  group  (FRG)  [5,  6,  7],  the  phase  diagram  of  the  extended  Hubbard  model 
was  still  missing.  Upon  different  filling  fractions,  on-site  U  and  nearest-neighbor  V  inter¬ 
actions,  we  found  several  instabilities,  including  the  well  known  charge(spin)-density  wave 
phase  at  half  filling,  dxy-wave  superconducting,  and  dx2_y2  pairing  state  under  slightly  dop¬ 
ing.  One  of  most  intriguing  results  is  triplet  time-reversal  breaking  {px  -t-  ipy)-wave  pairing 
instability  merges.  This  results  suggest  that  zero-energy  Majorana  modes  can  be  realized 
on  a  square  lattice.  Considering  a  system  of  cold  fermionic  atoms  on  a  two-dimensional 
square  optical  lattice,  both  an  on-site  repulsion  and  a  nearest-neighbor  attraction  would 
be  required,  in  addition  to  rotation  of  the  system  to  create  vortices.  We  discussed  possible 
ways  of  experimentally  engineering  the  required  interaction  terms  in  a  cold  atom  system. 

In  chapter  4,  we  apply  this  RG  idea  on  Hubbard  model  [8]  with  spin  imbal¬ 
anced  [9] .  Experiments  with  ultra-cold  atoms  have  realized  mixtures  of  two  different  species 
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of  fermionic  atoms  with  different  densities,  including  mixtures  of  cold  lithium  atoms  (®Li) 
with  different  populations  for  two  different  hyperfine  states  [10,  11].  With  the  presence  of 
square  optical  lattice,  we  can  model  such  a  system  as  a  Hubbard  model  where  the  spin-up 
and  spin-down  particle  has  different  densities.  Comparing  to  the  imbalanced  Fermi  gas 
(without  lattice),  the  lattice  effect  can  provide  much  richer  physics  and  accessible  critical 
temperature.  We  are  the  first  group  obtain  the  phase  diagram  for  this  system,  focusing 
on  the  case  where  the  majority  species  stays  close  to  half-filling  and  the  density  of  the 
minority  species  is  varied.  We  found  several  new  phases,  including  a  stripe-CDW  phase 
and  triplet  pairing  phases  for  both  species.  The  interplay  between  lattice  effect  and  the 
density-imbalanced  provides  important  features  for  quantum  correlated  system.  Also,  the 
critical  temperature  of  each  phase  is  experimentally  accessibility  and  much  higher  than  the 
one  without  lattice. 

In  chapter  5,  we  proposed  and  studied  a  quantum  degenerate  Fermi-Fermi  mixture 
model  on  square  lattice  [12].  According  to  recent  experiments,  a  system  with  different  types 
of  quantum  degenerate  fermionic  atom  can  be  engineered,  for  instance,  ®Li  and  [13]. 
In  this  two-species  Hubbard  model,  both  inter-species  and  intra-species  interactions  are 
taken  into  account  at  equal  footing.  Unlike  the  usual  single-species  (orbital)  model,  the 
competition  between  inter-species  and  intra-species  interaction  provides  more  intriguing 
strongly  correlated  effect.  Our  study  shows  the  various  dominant  instabilities  under  differ¬ 
ent  interaction  ratios  and  the  number  density  imbalance  between  different  atoms.  One  of 
exciting  result  is  the  unconventional  dxy-wave  charge  density  wave,  which  lives  in  a  very 
wide  parameter  regime  of  the  phase  diagram  and  has  an  accessible  critical  temperature. 
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Most  importantly,  our  theory  provides  a  general  screening  mechanism  of  forming  this  kind 
of  exotic  phase  on  a  square  lattice.  This  study  sheds  the  light  on  searching  unconventional 
density  wave  for  a  minimum  on-site  Hubbard  model. 

In  chapter  6,  we  use  FRG  method  to  study  a  dipolar  fermion  system  on  a  square 
lattice  of  bilayer.  The  recent  experiment  is  able  to  prepare  electric  [94]  or  magnetic 
dipoles  [93]  in  a  magneto  atomic  trap.  The  former  can  also  be  loaded  onto  optical  lat¬ 
tice  [117].  The  intriguing  long-range  interaction  already  draw  a  lot  of  attention  on  theoret¬ 
ical  studies  [121,  122,  123,  124,  125,  126,  127].  On  a  two  dimensional  square  lattice,  this 
kind  of  system  was  studied  before  [83,  100]  as  well.  In  the  two  dimensional  plane,  the  phase 
diagram  can  be  interpreted  by  two  critical  angles  [83].  Also,  this  long-range  anisotropic 
interaction  drives  the  system  into  some  unconventional  density  and  pairing  ground  states 
when  the  dipoles  are  oriented  in  particular  polar  angles.  Due  to  the  anisotropic  nature  of 
dipole-dipole  long-range  interaction,  the  presence  of  second  layer  brings  up  the  interplay 
between  intra-  and  inter-layer  interactions,  which  makes  this  system  more  interesting  than 
monolayer  system.  Also,  the  unconventional  charge  density  wave  is  found  in  wilder  range 
in  phase  diagram  than  one  in  monolayer  square  lattice. 

Conclusions  and  outlook  are  given  in  chapter  7.  The  back-matter  contains  ap¬ 
pendix  and  the  bibliography. 
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Chapter  2 

Functional  Renormalization  Group 
for  Interacting  Fermion 

2.1  Renormalization  Group  Method 

The  functional  renormalization  group  approach  to  interacting  fermions  [3,  5,  6]  has 
been  applied  to  the  study  of  the  stability  and  instabilities  of  Fermi  liquids.  Starting  with 
a  microscopic  Hamiltonian,  the  renormalization  group  method  provides  an  effective  theory 
for  low-energy  scales  by  integrating  out  high-energy  degrees  of  freedom,  reducing  the  UV 
cutoff  A  =  Aoe“^.  The  starting  point  of  this  approach  is  the  many-body  partition  function 

Z  =  f ,  (2.1) 

7 

where  T,  T  are  Grassmann  fields  with  auxiliary  index  7,  including  frequency,  momentum, 
orbital  and  spin  variables.  These  fields  are  divided  into  high  (T>)  and  low  ('f<)  energy 
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modes  depending  on  whether  their  energies  are  higher  or  lower  than  the  energy  cut-off  A, 
and  the  action  can  then  be  written  as 


5{^<,  ^'>1  =  So{^<}  +  5o{^'>}  +  '!'>}  ,  (2.2) 

where  Sq  is  the  non-interacting  quadratic  part  and  Sj  contains  quartic  interaction  terms. 

The  effective  low-energy  action  containing  only  low-energy  modes  is  obtained  by 
tracing  out  high-energy  modes  and  is  given  by 


5a  =  In  j  .  (2.3) 

This  procedure  is  carried  out  in  infinitesimal  successive  steps,  A  — )■  A'  =  As  shown 

in  Fig.  2.1,  the  recursive  steps  of  integrating  out  high  energy  modes  result  in  a  serious  of 
effective  Hamiltonian  at  different  RG  scale  i.  Back  to  the  formulation  of  partition  function, 
we  have 


Z  = 


The  effective  action  S'  is  given  by 


=  ^ 


(2.4) 

(2.5) 


(2.6) 
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k— space 


Figure  2.1:  The  recursive  steps  of  integrating  out  high  energy  modes  push  the  cut-off  Ai 
toward  Fermi  level  ep.  This  systematic  procedure  results  in  flow  of  effective  Hamiltonian 


,  where  the  constant  term  (Zo>  =  is  drop  in  last  line.  We  rewrite  above 

equation  into 

.  (2.7) 

Then  we  expand  it  by  means  of  cumulant  expansion  to  get 

6S'  =  {5S)  +  ^{{6S^)-{6Sf)  + .  (2.8) 

,  where  the  5S  is  linear  in  four-point  vertex  ([/). 

Here,  we  present  an  example  with  SU{2)  symmetry  [6,  7].  We  calculate  the  cor¬ 
rections  to  the  interaction  vertices  to  one- loop  level  [6].  During  this  RG  process,  we  do 
not  assume  any  prior  order  parameters.  Both  particle-particle  (pp)  diagram  in  Fig.  2.2b 
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Figure  2.2:  (a)  A  four-point  vertex  function  U{ki,  k2,  k^,  k^  =  ki  +  k2  —  k^).  (b)-(f)  One- 
loop  diagrams  that  contribute  to  the  renormalization  of  a  four-point  vertex. 


and  particle-hole  (ph)  diagrams  in  Figs.  2.2c-2.2f  are  included.  Therefore,  entire  process  is 
unbiased  to  any  order  state.  We  show  all  one-loop  Feynman  diagrams  in  Figs.  2.2b-2.2f, 
and  corresponding  RG  equation  is  given  by  following 

deU{£)  =  /3i{U}  =  /?pp{[/,  [/}  +  2/?ph{[/,  [/} 

-/3ph{17,  XU}  -  MXU,  U}  -  X^XU,  XU}  .  (2.9) 

The  function  /3  is  a  vertex  object  and  a  bilinear  functional  of  four-point  vertex  at  different 
RG  scale,  U{i).  The  operator  X  acting  on  a  four-point  vertex  object  exchange  its  in¬ 
dices,  saying  XF{1,  2,  3,  4)  =  F(2, 1,  3,  4).  /3pp  is  the  particle-particle  part  of  beta  function, 
diagrammatically  shown  in  Fig.  2.2b,  and  it  is  given  by 

(3pp{U,  U}  =  E{U,  U}  +  E{XU,  XU}  (2.10) 

2/3ph{C/,  17}  is  the  particle-hole  diagram  with  Fermi  bubble  in  Fig.  2. 2d,  and  it  is  worthy  to 
mention  that  —2  from  Fermi  bubble  is  included  here.  The  particle-particle  beta  function  is 
given  by 

MUu  U2}  =  n{C/i,  U2}  +  TU{Uu  U2}  (2.11) 
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,  with  the  time  inversion  operator  T-F(l,2,3,4)  =  F(3,4, 1,2).  Both  H  and  11  four-point 
functions  are  the  on-shell  integrals  of  particle-particle  and  particle- hole  diagrams.  More 
explicitly,  it  can  be  written  in  integral  form  with  two  fermion  propagators  (also  known  as 
Green’s  function)  as  following 

E{U,U}{K,,K2,K,,K,)  =  x  (2.12) 

T  J]G,(KH)G,^p(-i^(.)  +  Qpp)  X  t/(i^i,i^2,i^H,4p)t^(^3,i^4,i^H,^pp) 

LiJn 

n{t/i,C/2}(i^l,K2,i^3,i^4)  =  7^  [  ^0(|6fc,+,^J-A,)x  (2.13) 

T  +  Qph)  X  t/i(i^i,K(,),K3,4h)t^2(i^4,A:H,K2,  V)  • 

U)rL 

The  summation  over  index  z/  =  ±  is  over  inner  shell  (below  Fermi  level)  and  outer  shell 
(above  Fermi  level).  We  sketch  it  in  Fig.  2.1.  The  integral  is  along  a  constant  energy  curve 
e  =  vh.,  and  Vy  are  on-shell  velocities.  The  symbol  Ky  contains  momentum  (fc),  frequency 
(wn),  and  orbitals  (o)  etc..  The  quantity  inside  those  propagators  are 

Qpp  —  (QpP)  ^n,pp)  Oq)  —  Ki  K2  (^'1^) 

Qph  ~  {Qphi  ^n,phi  Oq)  =  Kl  K3  ■  (2.15) 

Next,  we  are  going  to  neglect  the  self-energy  correction  since  it  is  just  a  constant  which 
can  be  absorbed  into  chemical  potential  in  one- loop  manner  [6].  Thus,  we  can  use  bare 
Fermi  propagator  in  above  equations:  G(  ^  Gp  =  (iuj  —  e^)”^.  According  to  Shankar’s 
argument  at  tree  level  [3],  marginal  four-point  vertex  is  independent  of  the  magnitude  of 
momentum  and  frequency.  Therefore,  we  can  simplify  two  vertices  in  terms  of  angles, 
instead  of  auxiliary  indecies: 

C/(iFi,  Ar2,  it's,  ^)  — )■  t/(0i,  6*2,  03,  ^),  :4;i^2,3,4  =  0,  and  1,  2, 3, 4  on  Fermi  surface.  (2.16) 
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Next,  we  assume  zero  temperature  (T  — )•  0),  and  perform  Matsubara  summation  [6]  on 
Eq.  2.12.  In  order  to  solve  the  RG  flow  equations  numerically,  we  discretize  the  Fermi 
surfaces  [6,  5]  into  finite  number  of  patches  and  label  the  interaction  vertices  in  terms  of 
patch-indices,  instead  of  angles:  U{6i,62,03,i)  — )■ 

For  the  multi-species  system  we  consider  in  following  chapters,  each  solid  line 
in  these  diagrams  also  carries  a  spin  and  a  species  index.  If  the  Hamiltonian  has  SU{2) 
symmetry  for  each  species,  it  is  enough  to  calculate  only  vertices  in  which  fermions  with 
antiparallel  spins  interact,  as  vertices  involving  fermions  with  the  same  spin  can  be  obtained 
from  these  by  imposing  SU{2)  symmetry [6].  Depending  on  the  species  indices,  we  have 
intra-species  interaction  vertices  for  each  species  and  also  inter-species  vertices.  On  the  other 
hand,  if  system  Hamiltonian  also  has  global  17(1)  symmetry  and  interactions  are  density- 
density  type,  to  each  fermion  annihilated  (incoming  fermion  line  in  Fig.  2.2)  corresponds  a 
fermion  with  the  same  spin  and  species  indices  being  created  (outgoing  line).  Momentum 
conservation  requires  fci  -|-  ^2  =  ^3  +  ^4  in  every  vertex.  Also,  the  species  indices  appear  as 
a  pair  with  the  same  spin;  this  follows  from  the  density-density  nature  of  the  inter-species 
interaction  in  the  Hamiltonian. 

2.2  RG  Flow  Analysis 

After  carrying  out  the  one-loop  RG  calculations,  we  obtain  the  flow  of  four-point 
vertices  in  terms  of  the  RG  scale  i,  and  three  independent  momenta,  U{ki,k2,k3,i).  The 
momenta  are  discretized  in  the  numerical  implementation  but  we  describe  here  the  general 
analysis  in  terms  of  momenta.  The  signature  for  the  occurrence  of  an  instability  of  the  Fermi 
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liquid  state  in  the  RG  approach  is  the  development  of  run-away  flows.  In  order  to  determine 
the  dominant  instability,  we  look  at  the  quartic  interaction  terms  in  the  effective  action  in 
the  form  {k,p)0]^0p,  with  Ok  being  a  term  bilinear  in  the  fermion  fields  and 

corresponding  to  the  order  parameter  (OP)  of  pairing  (SC),  charge  density  wave  (CDW), 
spin-density  wave  (SDW),  ferromagnetism  (FM)  or  Pomeranchuk  (PI)  instabilities[6,  14]. 
Detail  definitions  of  order  parameters  are  given  in  appendix.  The  choice  of  order  parameters 
depends  on  different  symmetry  property  of  Hamiltonian.  We  discuss  three  different  types 
of  symmetry  in  the  following. 

2.2.1  U{1)  symmetry 

The  straightforward  case  is  the  spinless  fermion,  which  only  has  17(1)  symmetry. 
The  order  parameters  we  can  have  are  spin-triplet  SC,  CDW  and  PI.  The  corresponding 
matrices  are  related  to  U  by 


Vpl{k,p) 

where  k  =  k  +  Q,p  =  p  +  Q,  and  Q 
momenta,  k,  p,  lie  on  Fermi  surface. 


1 

1 

(2.17) 

U{k,p,k,p,i) 

(2.18) 

U{k,p,k,p,e)  , 

(2.19) 

the  nesting  vector  at  particular  filling,  and  the 
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2.2.2  SU{2)  symmetry 

If  system  has  SU{2)  symmetry,  we  can  write  most  general  interaction  form  into 
spin-spin  and  charge-charge,  or  singlet-singlet  and  triplet-triplet  interactions.  The  corre¬ 
sponding  matrices  are  related  to  ?7  =  by 


=  \  [U{k,-k,p,-p,£)  ±  U{-k,k,p,-p,i)] 

(2.20) 

=  -Uip,k,k,p,e) 

(2.21) 

'^CDw(^’P) 

=  ‘^U{k,p,k,p,i)  -U{p,k,k,p,i) 

(2.22) 

=  -Uip,k,k,p,i) 

(2.23) 

VpJ(fc,p) 

=  ‘^U{k,p,k,p,i) -U{p,k,k,p,£)  , 

(2.24) 

The  singlet  (triplet)  SC  instability  channel  is  denoted  by  SCsiSCt^)-,  and  corresponds  to 
the  plus(minus)  sign  in  the  right-hand  side  of  the  first  equation  above.  In  the  numerical 
implementation,  each  Fermi  surface  is  discretized  into  patches  and  k,  p  can  be  labeled  by 
a  discrete  patch  index  that  goes  from  1  to  M  (total  patch  number) . 

In  multi-species  system,  the  couplings  Vop{k,p)  contain  both  intra-  and  inter¬ 
species  couplings,  depending  on  whether  the  momenta  k  and  p  are  on  the  same  Fermi 
surface  or  on  different  Fermi  surfaces,  and  thus  k  and  p  have  M  values.  Explicitly,  each  V 
matrix  is  block  diagonal  for  intra-species  and  block  off-diagonal  for  inter-species.  For  each 
order  parameter  channel,  we  can  further  diagonalize  Vop{k,p)  into 

vW(fe,p)  =  Y,^^^ii^)VofS*ik,o,i)rjo>fg\p,o',£)  ,  (2.25) 

j,oo' 
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Figure  2.3:  An  example  of  RG  flows.  We  only  plot  the  leading  eigenvalue  in  each  channels. 

with  j  being  a  decomposition  index,  and  where  now  we  explicitly  indicate  the  species  index 
o  and  d  for  different  types  of  species.  In  Fig.  2.3,  we  show  an  example  of  RG  flow  which 
determines  an  instability.  The  leading  instability  can  be  determined  by  the  most  negative 
eigenvalue  (largest  magnitude),  and  the  corresponding  symmetry  of  the  instability 

(s-,  p-,  and  d-wave  etc.)  is  given  by  the  form  factor  The  sign  structure  factor 

r]o  =  1(~1)  stands  for  in-phase  (out-phase)  between  species  on  lattice. 

2.2.3  Broken  SU{2)  symmetry 

In  order  to  analyze  the  RG  flow  in  this  non-Sf7 (2)  symmetry  case,  we  need  to  derive 
the  instability  channels  from  its  original  dehnition,  which  is  shown  in  appendix.  Also,  we 
need  to  keep  spin  indices  all  the  time.  On  the  other  hand,  we  still  keep  global  U{1)  symmetry. 
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As  a  consequence  of  SU{2)  symmetry  broken,  the  spin-spin  interactions  decompose  into 
longitudinal(5DW2)  and  transpose  direction(S'Z9Vka;j^).  The  order  parameters  in  density 
wave  channels  are 

Uii{p,k,k,p,e)  +  U^^{k,p,k,p,tj  +  C4-f-(fc,p,fc,p,£)  (2.26) 

=  U^^{^,p,k,p,£)-U^^{p,k,k,p4)  +  U^{k,p,k,p,t}- 

Uii{p,k,k,p4)  -  U^;i{k,p,k,p,t)  -  (2.27) 

■  (2-28) 

For  pairing  order  parameters,  three  triplet  states(to  and  t±)  are  no  longer  degenerate  when 
SU{2)  symmetry  is  broken. 


'^SCs^^’P') 

'*25Ct+(^’P) 
'^SCt-  i^’P) 


\  Pnik>  -k,p,  -p,i)  +  C4t(^,  -k,p,  -p,i)+ 

Uni-k,  k,p,  -p,  €)  -k  k,p,  -p,  i)] 

\  [Uuik,-k,p,  -p,i)  -  Ui^{k,-k,p,  -p,£)+ 
Uui-k,k,p,  -p,£)  -  Ui^{-k,k,p,  -p,i)] 

\  [Unik,-k,p,  -p,i)  -  U^'f{-k,k,p,  -P,£)] 

\  [Uuik,-k,p,  -p,£)  -  Uii{-k,k,p,  -p,£)\ 


(2.29) 


(2.30) 

(2.31) 

(2.32) 


If  we  apply  the  SU{2)  symmetry  properties  that  Uaaiki,  ^2,  ^4)  =  ^{Uaa{ki,  ^2,^3,  ^4)  — 

Uaa{k2,  ki,  k^,  k/^)) ,  the  above  definitions  coincide  with  those  definitions  in  2.2.2. 
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Chapter  3 

2D  Extended  Hubbard  Model  on 
Square  Lattice 

3.1  Introduction 

An  extended  Hubbard  model  is  generally  employed  as  a  theoretical  framework 
of  screened  electronic  interactions  and  regarded  as  a  prototypical  scenario  for  rich  quan¬ 
tum  phases  in  condensed  matter  physics  [15,  16,  17].  In  a  one-dimensional  chain,  for  in¬ 
stance,  an  extended  Hubbard  model,  including  an  on-site  and  nearest-neighbor  interac¬ 
tions,  presents  correlated  phases  associated  with  the  ratio  of  the  two  interactions  [18,  19]. 
Recent  identification  of  the  bond-charge-density-wave  instability  between  charge  and  spin 
density-wave  phases,  at  weak  and  strong  interactions,  completes  the  phase  diagram  of  this 
model  [20,  21,  22,  23,  24,  25].  Meanwhile,  in  a  two-leg  ladder,  a  checkerboard  charge- 
ordered  state  has  been  proposed  for  all  fillings  between  quarter  and  half,  with  on-site  and 
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nearest-neighbor  repulsion  [26].  Its  application  to  the  coupled  quarter-filled  ladders  with 
coupling  to  the  lattice  has  recently  been  studied  [27]  to  explain  the  spin  gaps  in  the  NaV205 
material  [28,  29]. 

On  a  two-dimensional  lattice,  the  extended  Hubbard  model  has  been  considered  a 
paradigmatic  model  to  search  for  possible  unconventional  superconducting  phases  since  the 
discovery  of  high-temperature  superconductivity  in  the  cuprates  [30] .  Although  a  nearest- 
neighbor  repulsion  between  electrons  suppresses  non-s-wave  pairings  tendencies  [31],  it  is 
generally  believed  that  the  nesting  of  the  Fermi  surface  plays  a  key  role  in  driving  unconven¬ 
tional  pairing  under  purely  repulsive  interactions  at  weak  couplings  [18,  32,  33,  7,  34,  35,  36]. 
In  the  proximity  of  density-wave  order,  for  instance,  a  chiral  d-wave  state  has  been  found  for 
an  extended  Hubbard  model  on  both  triangular  and  honeycomb  lattices  [37,  38,  39].  Fur¬ 
thermore,  following  the  recent  experimental  realization  of  a  two-dimensional  Kagome  lattice 
for  ultracold  atoms  [40],  the  phase  diagram  of  the  extended  Hubbard  model  on  a  Kagome 
lattice  has  been  established  in  the  vicinity  of  van  Hove  fillings  [41].  It  was  shown  that  a 
possible  p-wave  charge  and  spin  bond  order  can  be  triggered  in  the  presence  of  a  nearest- 
neighbor  repulsion  for  van  Hove  fillings,  then  giving  way  to  a  /-wave  superconducting  phase 
when  slightly  doped  away  [39]. 

It  has  recently  been  proposed  that  under  a  nearest-neighbor  attraction  and  on-site 
repulsion  a  singlet  {p  -f  ip)-wave  pairing  emerges  on  a  honeycomb  lattice  [42].  Along  with 
the  result  that  Majorana  fermions  can  be  generated  as  a  zero-energy  mode  in  the  excitation 
spectrum  of  a  half-quantum  vortex  m.  a  {p  +  ip)-wave  superconductor  [43],  this  indicates 
the  possibility  of  creating  Majorana  fermions  in  graphene  in  the  presence  of  a  magnetic 
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field.  However,  a  functional  renormalization  group  study  has  shown  that  for  a  honeycomb 
structure,  the  /-wave  pairing  is  preferred  and  is  stabilized  by  introducing  a  next-nearest- 
neighbor  attraction  [37].  Although  triplet  p-wave  superconductivity  has  been  proposed  for 
an  extended  Hubbard  model  on  a  square  lattice  for  purely  repulsive  interactions  [44,  45],  the 
physics  of  long-range  interactions,  with  possibly  terms  leading  to  competing  instabilities, 
has  been  studied  recently,  leading  that  the  existence  of  p-wave  pairing  is  still  an  open 
question  [46,  34,  35]. 

3.2  Model  and  Formalism 

In  this  chapter,  we  study  the  phase  diagram  of  an  extended  Hubbard  model  via  a 
functional  renormalization  group  (FRG)  approach  [3,  5,  6],  including  on-site  U  and  nearest- 
neighbor  V  interactions,  on  a  two-dimensional  square  lattice.  The  total  Hamiltonian  can 
be  written  as,  H  =  Hq  +  77mt,  with  the  non-interacting  and  interacting  parts, 

Ho  =  -tY^  (cl^cja  +  H.c.)  (3.1) 

{ij),a  i 

Hint  —  U  -|-  V  ^  ^  JliTlj ,  (3.2) 

*  (b'> 

respectively,  where  (ij)  represents  nearest-neighbor  pairs  of  sites,  n*  =  ni^+riii  = 
and  ^  is  the  chemical  potential.  Without  the  nearest-neighbor  interaction,  the  phase  di¬ 
agram  in  a  FRG  analysis  is  well  developed  in  the  limit  of  weak  couplings  [5].  For  on-site 
attraction,  s-wave  superconductivity  (s-SG)  dominates  at  all  fillings  except  for  half-filling. 
The  phase  diagram  for  on-site  repulsion,  U  =  1,  versus  ^  is  sketched  in  Fig.  3.1.  In  the 
vicinity  of  half-filling,  the  spin-density  wave  (SDW)  dominates  due  to  the  strong  nesting  of 
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the  Fermi  surface.  With  slight  doping,  dx2_y2-SC  emerges  in  a  small  regime  of  just  away 
from  half-filling.  When  the  magnitude  of  //  is  further  increased,  no  instability  develops  in 
our  analysis,  up  to  the  point  where  we  stop  the  RG  flows,  at  energy  cutoffs  lower  than  10“®t. 
Kohn-Luttinger  (KL)  instability  [47]  is  expected  to  occur  in  this  regime.  Sufficiently  away 
from  half-filling,  the  particle-hole  channels  can  be  treated  perturbatively,  and  the  RG  flows 
at  low  energies  can  be  calculated  including  the  particle-particle  diagram  only  [6].  Previous 
works  have  found  that  the  d^2_y2-wsye  SC  phase  near  half-filling  is  followed  by  a  dxy-v^&SQ 
upon  doping  [6,  34,  35],  or  by  a  p-wave  SC  phase  that  is  then  followed  by  a  dxy-wave  SC 
phase  [44]. 

3.3  Results 

In  presence  of  a  nearest-neighbor  interaction  V,  the  phase  diagram  is  much  richer. 
We  find  that  dxy-  and  {px  +  ipy)-wave  pairing  superconducting  states  develop.  In  the 
proximity  of  the  charge-density-wave  (CDW)  order,  dxy-wave  pairing  emerges  from  the 
CDW  instability  with  U  >  0  and  P  >  0.  When  the  nesting  of  the  Fermi  surface  is  decreased, 
a  time-reversal  symmetry  breaking  {px  +  ipy)-SC  arises  from  the  dx2_y2-SC  with  a  sufficient 
large  nearest-neighbor  attraction.  When  nesting  is  completely  suppressed,  {px  4-  ipy)-SC 
dominates  in  the  whole  regime  of  1/  >  0  and  P  <  0,  and  the  dxy-paiTm.g  is  only  triggered  with 
the  help  of  an  on-site  attraction.  Using  a  symmetry  argument,  we  show  that  appearance 
of  {px  +  ipy)-SC  on  a  square  lattice  for  P  <  0  is  generic  and  robust  due  to  the  underlaying 
lattice  structure,  and  can  be  used  to  create  a  zero  mode  Majorana  fermion  in  the  presence 
of  a  magnetic  field. 
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Figure  3.1:  Fermi  surfaces  of  a  square  lattice  with  nearest-neighbor  hopping,  at  different 
chemical  potentials  /r  (in  units  of  t).  Below,  the  phase  diagram  versus  ^  for  17  =  1  and 
F  =  0.  The  Fermi  surface  patches  used  in  this  study  are  illustrated  in  the  Fermi  surface  at 

/i  =  0. 


3.3.1  At  and  near  half  filling 

Starting  with  the  bare  Hamiltonian,  Eq.  (3.2),  we  follow  standard  FRG  procedure 
integrating  out  high-energy  modes,  decreasing  the  energy  cut-off  A.  The  four-fermion  terms 
in  the  resulting  effective  Hamiltonian  are  written  in  the  form 


g{ki,  k2,  fcs,  A)V’i(fc3)V'^(fei  +  ^2  -  fc3)V’/3(^2)V'a(fci)  (3.3) 


,  in  momentum  space,  with  k  =  {kx,  ky)  and  spin  indices  a,  /?.  In  previous  studies,  the  RG 
equations  for  models  with  SU{2)  and  17(1)  symmetries  have  been  systematically  studied  by 
Fermi  surface  discrete  patch-approximation  [5] .  In  this  chapter,  to  preserve  the  particle-hole 
symmetry  of  the  non-interacting  Hamiltonian  at  half-filling,  all  phase  diagrams  are  obtained 
by  the  conhguration  of  the  Fermi  surface  patches  illustrated  in  Fig.  3.1.  By  integrating  out 
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Figure  3.2:  The  phase  diagram  (a)  at  half-filling  /i  =  0,  (b)  at  /r  =  —0.095,  parameterized 
by  on-site  U  and  nearest-neighbor  interaction  V.  The  interaction  terms  U  and  V  are  in 
units  of  t  throughout  this  chapter. 


high  energy  degrees  of  freedom  and  neglecting  self-energy  corrections,  the  RG  flows  of  all 
couplings  versus  the  decreasing  running  energy  cutoff  A  are  computed.  Before  the  system 
flows  into  the  strong  couplings  regime,  we  truncate  the  RG  process  when  the  absolute 
magnitude  of  one  of  the  couplings  reaches  ~  30t. 

To  determine  the  dominant  instability,  we  decompose  specific  four-fermion  interac¬ 
tion  terms  in  the  Hamiltonian  as  ^  Vop{k,  p,  A)Ol,Op,  with  Oj.  a  bi-fermion  operator  for 
the  order  parameter  (op)  of  SC,  CDW,  SDW  or  Pomeranchuk  instability.  Then,  for  a  given 
order  parameter  channel,  we  further  decompose,  Vop{k,p,  A)  =  -  t(;*p(A)/®p(fc,  A)/*p(p,  A), 

in  normal  modes,  with  i  a  symmetry  decomposition  index.  The  leading  instability  can  be 
determined  by  the  most  minimum  eigenvalue  r(;™“(A)  (largest  magnitude),  and  the  corre¬ 
sponding  symmetry  of  the  instability  is  given  by  the  form  factor  [6,  33]. 
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At  half-filling,  the  phase  diagram  has  been  studied  extensively  by  several  meth¬ 
ods  [48,  49,  50].  In  our  FRG  analysis,  we  include  the  regime  of  negative  interactions  and 
obtain  the  phase  diagram  in  Fig.  3.2a,  parameterized  by  on-site  interaction  U  and  nearest- 
neighbor  interaction  V  (in  unit  of  t).  The  phase  boundary  between  SDW  and  CDW  is  at 
U  ~  41/  for  U,V  >0,  consistent  with  known  results  from  previous  studies.  For  the  line  of 
F  =  0  and  U  <  0,  we  find  that  CDW  and  s-SC  are  degenerate,  also  in  agreement  with  re¬ 
sults  in  the  literature  [50].  However,  the  degeneracy  is  delicate  and  broken  by  introduction 
of  a  nearest-neighbor  interaction;  a  slight  nearest-neighbor  attraction  drives  the  system  to 
a  s-SC  instability,  instead  of  a  CDW  instability. 

We  also  find  that  for  a  sufficiently  large  nearest-neighbor  attraction,  a  d^2_y2-SC 
is  triggered,  even  dominant  over  the  s-SC  in  the  regime  of  U  <0.  This  d-wave  pairing 
is  linked  with  the  nesting  of  the  Fermi  surface.  In  other  words,  if  the  nesting  effect  is 
suppressed,  the  s-SC  is  the  dominant  instability  for  a  generic  Fermi  surface  in  the  regime  of 
U  <  0.  Furthermore,  without  nesting  effects,  a  p-wave,  with  lower  angular  momenta  than 
d-wave,  will  eventually  dominate  in  the  regime  of  F  <  0  and  U  >  0.  However,  negative  F 
combined  with  nesting,  leads  to  d^2_y2-SC. 

By  slightly  doping  away  the  half-filling,  the  da,2_y2-SC  overcomes  the  spin-density- 
wave  instability  that  dominates  for  on-site  repulsion.  The  phase  diagram  parameterized  by 
U  and  F  is  illustrated  in  Fig.  3.2b.  The  SDW  is  suppressed  to  the  small  regime  between 
CDW  and  d^2_y2-SC  in  the  phase  diagram.  The  degenerate  line,  U  <  0  and  F  =  0, 
mentioned  above,  is  dominated  by  s-SC  instability  since  the  Fermi  surface  is  not  perfectly 
nested.  However,  the  CDW  is  still  dominant  in  the  overall  regime  of  F  >0. 
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Figure  3.3:  Phase  diagrams  for  (a)  /r  =  —0.5,  (b)  ^  =  —  1  and  (c)  ^  =  —2,  parameterized 
by  U  and  V.  The  transparent  regime  means  no  instability  is  found  before  we  stop  the  RG 
process,  at  which  point  the  energy  cutoff  is  lower  than  Form  factors  obtained  from 

decoupling  of  pairing  channels  into  bi-fermions  during  the  FRG  flows  are  illustrated  for  (d) 
dxy-wave  (e)  {px  +  fpy)-wave  pairings. 

3.3.2  Doped  systems 

Here,  we  increase  doping,  decreasing  nesting  of  the  until  the  density-wave  insta¬ 
bility  no  longer  occurs.  Then,  as  shown  as  Fig.  3.3a,  the  dxy-wave  SG  arises  from  the  CDW 
instability  in  the  regime  of  R  >  0  and  ?7  >  0.  The  form  factors  of  the  dxy-SC  in  our 
FRG  analysis  is  plotted  in  Fig.  3.3d.  Although  the  dxy-SC  phase  has  been  proposed  for  the 
purely  repulsive  models  [44,  46,  34,  35],  the  dxy-SC  instability  we  find  develops  only  with 
an  appropriate  nearest-neighbor  repulsion  and  also  tied  with  the  proximity  to  nesting  of 
the  Fermi  surface.  It  is  only  when  nesting  is  barely  suppressed  so  that  CDW  is  no  longer 
dominant,  but  CDW  fluctuations  are  still  expected  to  be  strong,  that  this  dxy-SC  phase 
emerges. 
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As  compared  with  the  phase  diagram  at  p  =  —0.5  in  Fig.  3.3a,  the  dxy-SC  in¬ 
stability  moves  to  the  region  of  larger  positive  V  or  negative  U  when  doping  is  increased, 
for  fj,  =  —1,  as  shown  in  Fig.  3.3b.  By  looking  at  the  RG  flows  of  different  couplings, 
we  notice  that  the  different  behavior  for  the  two  fillings,  p  =  —0.5  and  p  =  —1,  are 


23 


38 


mainly  coming  from  the  effect  of  the  nesting  vertices,  that  is,  couplings  g{ki,k2,k^,k4) 
with  ki  +  k2  —  k^  —  k/j^  =  (0,  27r)  or  (27r,  0)  or  (27r,  27r).  For  //  =  —1,  there  are  fewer  nesting 
vertices  flowing  into  large  values  as  compared  with  the  =  —0.5  case.  Without  the  benefit 
of  nesting  vertices,  the  instability  of  dxy-  and  dx2_y2-SC  are  suppressed  and  eventually  do 
not  develop  by  the  time  we  stop  the  RG  flow.  As  a  consequence,  when  the  nesting  is  com¬ 
pletely  destroyed,  in  heavily  doped  cases,  there  is  no  instability  found  for  purely  repulsive 
interactions  {U  >  0  and  V  >  0),  as  shown  in  Fig.  3.3c. 

The  triplet  p-SC  phase  tells  an  opposite  story.  In  the  regime  of  F  <0,  under  the 
influence  of  Fermi  surface  nesting,  dx2_y2-SC  is  dominant,  as  shown  as  Fig.  3.3a.  However, 
decreasing  nesting  suppresses  the  d-wave  pairing  and  the  p-SC  emerges  for  large  nearest- 
neighbor  attraction,  as  shown  in  Fig.  3.3b.  When  the  nesting  is  no  longer  present  at  all, 
p-SC  dominates  the  regime  of  F  <  0  and  17  >  0,  as  illustrated  as  Fig.  3.3c.  The  emergence  of 
a  triplet  p-SC  has  a  clear  physical  picture  in  a  square  lattice.  In  the  presence  of  a  nearest- 
neighbor  attraction  F  <  0,  electronic  pairings  are  triggered.  To  lower  energy,  a  s-wave 
pairing  with  the  lowest  angular  momenta  would  be  favored.  However,  an  on-site  repulsion 
suppresses  the  s-wave  pairing  so  that  p-wave,  with  the  second  lowest  angular  momenta,  is 
preferred. 

Furthermore,  the  four-fold  symmetry  on  a  square  lattice  also  indicates  that  the  px- 
and  Py-wa,ve  instability  channels  must  be  degenerate.  The  degeneracy  of  two  p-wave  super¬ 
conducting  instabilities  is  indeed  found  in  our  FRG  results  and  the  associated  form  factors 
are  plotted  in  Fig.  3.3e.  Since  the  Px-  and  py-wave  superconducting  states  are  degenerate, 
the  gap  functions  will  be  given  as  a  linear  combination  of  the  two  order  parameters.  By 
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constructing  the  gap  function  as  (k)  +  vAp^  (k)  with  the  complex  coefficient  v 

containing  a  possible  relative  phase,  the  condensation  energy  in  the  standard  BCS  equation 
is  obtained  by  the  difference  in  energy  between  the  superconducting  and  normal  states,  and 
is  given  by 

AE  =  Esc  -  -En  =  2  ^2  - ^  (3,4) 

\k\>kp  _  ‘^\J 

where  is  the  dispersion  relation  of  the  non-interacting  Hamiltonian.  The  second  term  of 
Eq.  (3.4)  is  maximized  when  v  is  purely  imaginary,  hence  the  time-reversal  breaking  pairing 
symmetry  px  +  ipy  is  the  energetically  favored  one  [38,  39,  51,  52].  One  can  also  preform  the 
Ginzburg-Landau  free  energy  analysis  to  verify  that  px  +  ipy  is  preferred  in  energetics  [53] . 
Physically  this  is  reasonable,  since  this  choice  of  the  phase  guarantees  that  a  gap  forms 
everywhere  along  the  Fermi  surface,  lowering  the  ground-state  energy. 

3.4  Summary 

In  atomic  Bose-Fermi  mixtures,  an  effective  attraction  between  fermions  can  be 
mediated  by  fluctuations  of  the  Bose-Einstein  condensate  of  the  bosons  [54,  55,  56].  In  the 
presence  of  the  mediated  long-range  attraction,  the  px  +  ipy  wave  superconducting  state 
has  been  proposed  in  these  systems  [55].  However,  the  mechanism  discussed  here  puts  some 
constraint  for  the  emergence  of  the  px  +  ipy-SC  phase;  in  order  to  develop  the  px  +  ipy-SC, 
long-range  attraction  and  on-site  repulsion  is  needed,  as  well  as  low  density  of  fermions  to 
avoid  nesting  of  the  Fermi  surface,  while  large  enough  densities  such  that  the  Fermi  energy 
is  a  large  scale  {t  >  U,  V)  to  justify  the  validity  of  FRG  results.  Another  possible  way 
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to  manifest  a  mediated  attraction  is  through  another  species  of  fermions  in  a  Fermi-Fermi 
atom  mixture  [57].  By  introducing  an  inter-species  interaction  in  Fermi-Fermi  mixtures,  an 
effective  interaction  for  one  species  of  fermions  can  be  obtained  by  tracing  out  the  other 
species.  This  can  be  justified,  for  example,  if  the  one  of  the  species  has  a  much  smaller 
effective  mass  than  the  other.  In  this  case,  the  mediated  long-range  interaction  is  found  to 
decay  rather  rapidly  and  can  be  approximated  by  an  effective  on-site  and  nearest-neighbor 
interactions.  By  tracing  out  one  species  with  low  electronic  density,  an  effective  nearest- 
neighbor  attraction  can  be  obtained.  Together  with  a  bare  hard-core  on-site  repulsion,  it 
may  provide  the  required  conditions  for  the  creation  of  the  time-reversal  breaking  px  -|-  ipy- 
wave  pairing. 

In  conclusion,  we  study  the  phase  diagram  of  an  extended  Hubbard  model,  includ¬ 
ing  a  on-site  U  and  nearest-neighbor  V  interactions,  on  a  two-dimensional  square  lattice. 
In  the  proximity  of  charge-density-wave  order,  the  dxy-SC  overcomes  the  CDW,  dominating 
in  the  regime  of  U  >  0  and  F  >  0,  from  our  FRG  analysis.  Accompanying  the  destruction 
of  Fermi  surface  nesting,  a  time-reversal  breaking  (px  -|-  ipy)-wave  superconducting  state 
arises  in  the  regime  of  F  <0.  Our  results  indicates  that,  without  nesting,  the  {px  +  ipy)-SC 
on  a  square  lattice  under  a  nearest-neighbor  attraction  is  the  generic  behavior  due  to  the 
underlaying  lattice  structure,  and  can  be  used  to  create  a  zero  mode  Majorana  fermion  in 
the  presence  of  a  magnetic  field. 
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Chapter  4 

Spin-Imbalanced  Fermi-Fermi 
Mixture 

4.1  Introduction 

Experiments  with  ultra-cold  atoms  have  realized  mixtures  of  two  different  species 
of  fermionic  atoms  with  different  densities,  including  mixtures  of  cold  lithium  atoms  (®Li) 
with  different  populations  for  two  different  hyperhne  states  [10,  58,  59,  11,  60,  61,  62], 
Fermionic  systems  with  imbalanced  spin  populations  have  been  studied  in  electronic  mate¬ 
rials,  such  as  magnetic-held-induced  organic  superconductors  [63,  64].  Mixtures  of  different 
species  of  fermions  with  unequal  populations  have  also  been  considered  in  the  study  of 
quark  matter  [65].  With  the  rapid  experimental  advances  in  the  held  of  cold-atom  physics, 
these  systems  have  the  advantage  of  a  great  degree  of  tunability  and  control  of  inter-particle 
interactions,  dimensionality,  conhnement,  as  well  as  a  number  of  pseudo-spin  species.  Im- 
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balanced  mixtures  of  cold  fermionic  atoms  have  attracted  great  interest  due  to  their  possible 
rich  phase  diagram  [62],  Several  phases  have  been  observed  experimentally  [11,  62],  such  as 
imbalanced  superfluid  phase,  phase  separation,  and  normal  Fermi  liquid  behavior.  Also,  the 
Larkin-Ovchinnikov-Fulde-Ferrel  (LOFF)  state  [66,  67],  which  involves  Cooper  pairs  with 
finite  center-of-mass  momentum,  and  breached  pair  [68,  69,  70,  71]  state  with  zero  center- 
of-mass  momentum,  could  be  possible  phases  in  special  regions  of  the  phase  diagram  [72]. 
In  addition,  there  are  studies  of  p-wave  triplet  pairing,  caused  by  effective  attractive  intra¬ 
species  interaction,  proposed  for  strongly  imbalanced  cases  in  two  [73]  and  in  three  [74,  75] 
dimensions.  In  addition  to  being  cooled  and  trapped,  fermionic  atoms  and  mixtures  can  now 
also  be  loaded  onto  optical  lattices  [76,  77]  where  the  interaction  and  the  hopping  strengths 
can  be  tuned,  and  effects  of  their  interplay  with  lattice  geometry  and  dimensionality  can 
be  probed. 

We  consider  a  two-component  mixture  of  fermionic  atoms  with  imbalanced  popu¬ 
lations  on  a  two-dimensional  square  lattice  at  weak-coupling  region.  It  is  well  known  that 
the  presence  of  a  lattice  can  provide  interesting  strong  correlation  effects,  such  as  spin  den¬ 
sity  wave  (SDW)  and  charge  density  wave  (CDW)  phases  for  fermions  on  a  two-dimensional 
(2D)  square  lattice  at  half-filling.  Extensive  studies  can  be  found  in  the  literature  for  the 
case  of  balanced,  SU (2)-symmetric  fermions,  such  as  the  Hubbard  model  [8]  in  various  lat¬ 
tice  geometries  and  fillings  [3,  5,  6,  51].  Due  to  nesting  of  the  Fermi  surface  (FS),  SDW  is 
the  dominant  instability  for  the  repulsive  Hubbard  model  on  a  2D  square  lattice  at  half¬ 
filling,  followed  by  d^2_y2-wave  superconductivity  when  the  system  is  doped  away  from 
half-filling.  [5,  6]  When  nesting  is  completely  destroyed  by  doping,  the  system  becomes  a 
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normal  Fermi  liquid,  aside  from  Kohn-Luttinger  instabilities  [47]  at  extremely  low  temper¬ 
atures.  However,  when  the  spin  populations  are  unequal  and  SU(2)  symmetry  is  broken, 
both  the  SDW  and  the  singlet  pairing  will  eventually  be  precluded  due  to  the  mismatch 
of  the  FS  for  the  up-  and  down-spin  fermions  at  strong  population  imbalance.  As  the 
Fermi  surfaces  become  increasingly  mismatched,  the  system  is  expected  to  be  dominated 
by  other  instabilities  or  becomes  a  two  species  Fermi  liquid.  If  the  interspecies  interaction 
is  initially  repulsive,  one  expects  to  hnd  SDW  phase  near  the  balanced  case  [78,  79,  80], 
switching  to  other  and  potentially  richer  correlated  behavior  as  the  polarization  increases. 
Here,  we  study  this  behavior  using  a  weak-coupling  functional  renormalization  group  (FRG) 
method,  [3,  5,  6]  which  is  able  to  treat  different  instabilities  on  an  equal  footing.  We  obtain 
the  phase  diagram  for  this  system,  focusing  on  the  case  where  the  majority  species  stays 
close  to  half-filling  and  the  density  of  the  minority  species  is  varied.  The  phase  diagram 
(Fig.  4.1)  contains  several  new  phases,  including  a  stripe-CDW  phase  and  triplet  pairing 
phases  for  both  species.  From  our  FRG  study,  we  also  obtain  estimates  for  the  critical 
temperature  for  the  different  instabilities. 

For  a  microscopic  interaction  which  is  on-site,  there  is  no  bare  intra-species  in¬ 
teraction  due  to  Pauli  exclusion  principle,  but  an  effective  long-range  interaction  can  be 
induced  via  scattering  between  species.  Previous  studies  [73,  74,  75]  have  considered  such 
mediated  interactions  for  mixtures  of  fermion  gases  (no  lattice)  with  imbalanced  popu¬ 
lations,  finding  an  attractive  effective  intra-species  pairing  interaction,  leading  to  p-wave 
pairing  of  the  majority  species.  In  this  study,  we  consider  the  effects  of  the  interplay  of 
interaction,  population  imbalance,  and  lattice  effects.  We  show  that  lattice  effects  in  par- 
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ticular,  not  only  lead  to  a  much  richer  phase  diagram  than  that  of  an  imbalanced  mixture 
of  fermionic  gases,  but  also  with  a  much  higher  transition  temperature,  even  at  weak  in¬ 
teraction  couplings,  therefore  more  easily  accessible  to  experimental  observation.  Unlike 
Bose-Fermi  mixtures  [54,  55,  81,  82],  where  different  pairing  and  density  wave  states  for 
the  fermions  originate  from  attractive  interactions  mediated  by  quantum  fluctuations  of 
the  boson  condensate,  in  the  fermion  mixture  considered  here,  both  species  have  screening 
effects  from  each  other,  and  the  low  energy  physics  depends  on  the  interplay  between  initial 
interspecies  interaction,  induced  intra-species  interaction,  the  FS  geometry  of  each  species, 
and  their  mismatch  due  to  imbalance. 

4.2  Model  and  Formalism 

We  consider  a  one-band  Hubbard  model  for  each  species  (a  =t,  i)  of  fermion  (with 
creation  operator  on  a  2D  square  lattice,  with  on-site  interspecies  interaction  Uq-  The 
Hamiltonian  can  be  written  as: 


(jk  kk'q 


,  where  =  —2to-{cos  kx  +  cosky)  +  fia  and  V  is  the  volume  of  the  system  (hereafter  set 
to  be  equal  to  1).  The  different  chemical  potentials  /j,o-  determine  the  densities  and  the 
hopping  amplitude  can  be  tuned  by  the  optical  lattices.  In  this  work,  we  only  consider 
the  case  if  =  tj,  =  t,  and  weak  repulsive  interspecies  interaction  ([/q  >  0). 
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4.3  Results 

4.3.1  Nearly  balanced 

In  order  to  maximize  lattice  effects,  we  focus  on  the  case  where  the  majority  species 
is  at  or  close  to  half-filling,  ^  ~  0,  and  we  vary  Phase  diagrams  are  shown  in  Fig.  4.1. 
When  the  system  is  balanced  =  //^  =  0),  SDW  is  the  dominant  instability  (RG  flows 
shown  in  Fig.  4.2(a)).  With  increasing  the  SDW^  order  persists  for  the  nearly  balanced 
region  (Fig.  4.1),  but  the  critical  temperature  Tc  decreases.  This  result  agrees  with  previous 
studies  [78]  showing  that  antiferromagnetic  order  is  suppressed  by  increasing  imbalance  (i.e. 
A^). 

4.3.2  Weak  imbalance 

As  the  minority  density  is  further  decreased  away  from  half-filling,  some  inter¬ 
species  vertices  involving  nesting  vectors,  such  as  Umklapp  processes,  are  no  longer  near 
the  FS  and  are  thus  suppressed. 

However,  there  are  still  some  non-perfect  nesting  particle-hole  processes  with  net 
momentum  equal  to  a  reciprocal  lattice  vector,  (27r,0)  or  (0,  27r),  which  are  allowed  even 
under  imbalance.  An  example  of  such  a  process,  depicted  in  Fig.  4.2(b),  is  for  a  minority 
fermion  to  scatter  from  q  ^  q  +  Q2  —  S  across  two  opposite  sides  of  its  FS  (say  along  the 
45°  direction),  while  a  majority  fermion  scatters  from  k  ^  k  +  Qi  -|-  S  across  the  other  two 
opposite  sides  of  its  FS  (say  along  the  135°  direction).  With  Qi  =  (7r,7r),  Q2  =  (— 7r,7r), 
and  S  =  (S,  5)  accounting  for  the  FS  mismatch,  the  net  momentum  for  this  process  is 
(0,  27r)  and  therefore  of  Umklapp-type  and  allowed  by  momentum  conservation  in  spite  of 
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the  imbalance.  These  vertices  can  still  renormalize  significantly  under  RG  flow,  and  more 
importantly,  they  mediate  intra-species  nesting  processes  for  the  majority  fermions,  which 
do  have  a  perfectly  nested  FS,  so  these  mediated  processes  have  the  strongest  flow.  On 
the  other  hand,  the  induced  intra-species  long-range  attraction  is  maximum  at  diagonal 
directions.  Due  to  the  partial  nesting  and  intra-species  long-range  attraction,  this  leads  to 
a  crossover  in  the  RG  flows  from  the  SDWz  to  a  stripe-CDW  phase.  From  a  real-space 
picture,  with  increasing  imbalance,  both  species  also  have  more  empty  sites  to  move  around 
to  minimize  free  energy.  Figure  4.2(a)  shows  the  RG  flow  close  to  this  crossover,  where 
SDW2  and  GDW  channels  are  almost  degenerate.  The  GDW  channel  of  both  species  is 
doubly  degenerate.  The  order  parameters  for  '[  and  are  non-zero  on  two-opposite  sides 
of  the  FS  (second  and  fourth  quadrants),  and  zero  on  the  other  sides  (first  and  third 
quadrants),  with  the  situation  reversed  for  the  other  degenerate  channel  [Fig.  4.2(c)  shows 
one  of  the  cases)  These  correspond  to  stripe  charge  order  in  the  diagonal  direction  [83]). 
This  stripe-GDW  phase  originates  from  weaker  spin  fluctuations  caused  by  imbalance  in 
addition  to  screened  intra-species  long-range  attraction.  Since  it  involves  both  spin  and 
charge  fluctuations,  the  size  of  this  region  and  its  critical  energy  scale  in  phase  diagram 
depend  on  the  filling  fraction  of  both  species,  shown  in  Fig.  4.1. 

4.3.3  Strong  imbalance 

As  the  imbalance  becomes  stronger,  the  FS  mismatch  precludes  all  nesting  pro¬ 
cesses  as  well  as  zero-momentum  singlet  pairing.  Since  the  induced  intra-species  interactions 
are  always  attractive,  the  dominant  RG  flows  present  another  crossover,  from  GDW  stripe 
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phase  to  triplet  p-wave  BCS  pairing.  The  triplet  pairing  is  for  a  single  fermion  species  but 
it  is  generated  by  the  initial  bare  repulsive  interspecies  interaction,  the  lattice  (FS  shape), 
and  the  imbalance  (FS  mismatch).  Whether  the  dominant  pairing  is  for  the  majority  or 
for  the  minority  species  depends  on  the  interplay  of  these  factors.  Figure  4.3(a)  shows  RG 
flows  for  three  different  minority  chemical  potentials.  As  increases,  the  minority  pairing 
channel  is  dominant  at  first.  This  is  because  the  nearly  half- filled  majority  FS  provides  a 
large  phase  space  in  RG  process  to  mediate  and  renormalize  the  minority  intra-species  BCS 
vertex.  The  bubble  diagram  in  Fig.  2.2(d),  which  contains  an  internal  fermion  loop,  gives 
a  first  nonzero  correction  to  the  intra-species  vertex  from  two  interspecies  vertices,  of  the 
form. 


diUfl,{ki,k2,kz)  = 


PphiUiM,  (4.2) 

where  X  denotes  the  operation  AF(1,  2, 3, 4)  =  F(2, 1, 3, 4)  and  j3ph{Ui,  U2}  =  n{17i,  U2}  + 
7n{C/i,C/2},  with  TT(1,2,3,4)  =  F(3,4, 1,2)  the  time-reversal  operator  and 


n{t/i,t/2}  =  Bph{ki,  ks,  q)Uaaiki,  q,  k3)U^,a{k4,  q,  k2)  (4.3) 

q,a 

where  Bph  is  an  integral  over  an  angular  sector.  [6]  From  Eq.  (4.3),  the  induced  intra¬ 
species  BCS  vertex  comes  from  integrating  out  two  interspecies  interactions.  For  example, 
in  Fig.  4.3(b),  integrating  pairs  of  solid  and  dashed  lines,  which  represent  two  interspecies 
vertices,  results  in  an  intra-species  BCS  vertex,  Uii{k,—k,k',—k').  Because  the  FS  of 
the  majority  species  is  flat,  there  will  be  more  renormalization  corrections  to  intra-species 
interaction  between  minority  species  at  first.  As  shown  in  Fig.  4.3(c),  the  momentum  q 
at  fixed  magnitude  can  be  translated  anywhere  along  that  branch  of  the  FS.  Figure  4.1 
shows  that  the  critical  temperature(a.u.)  for  minority  pairing  decreases  monotonically  with 
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increasing  polarization,  because  the  phase  space  of  induced  minority  intra-species  pairing 
becomes  smaller.  The  same  effect  is  observed  for  the  majority  p-wave  pairing  channel,  which 
is  the  sub-leading  channel.  However,  the  induced  BCS  vertex  of  majority  species  couples 
to  its  own  nesting  channel  [k  and  k' ,  for  example,  are  connected  by  a  nesting  vector  in  Fig. 
4.3(b)].  Although  the  initial  mediated  majority  intra-species  pairing  interaction  is  smaller 
than  that  for  minority  species,  as  discussed  above,  eventually  the  strong  RG  flow  through  a 
nesting  channel  leads  to  a  majority  pairing  instability.  We  note  that  this  majority  superfluid 
is  not  the  same  as  proposed  in  previous  studies  of  fermion  gases  [73,  74,  75],  but  is  instead 
a  lattice  effect  and  has  a  much  larger  energy  scale  and  critical  temperature.  As  the  density 
of  minority  species  is  further  decreased,  it  starts  to  behave  like  a  fermion  gas,  with  a  small, 
almost  circular  FS,  as  shown  for  example  in  Fig.  4.3(c).  When  the  density  of  minority 
species  becomes  less  than  quarter  filling,  the  induced  majority  BCS  vertices  are  no  longer 
coupled  to  its  own  nesting  vertices,  which  causes  a  sudden  drop  of  the  critical  temperature 
for  the  majority  pairing  channel.  Since  the  majority  species  is  still  perfectly  nested,  there 
is  more  phase  space  for  the  minority  pairing  interactions  to  be  mediated  and  renormalized. 
This  crossover  of  RG  flows  is  shown  in  Fig.  4.3(a).  The  majority  pairing  instability  only 
exists  in  a  small  region  of  the  phase  diagram  for  each  different  majority  filling,  as  shown  in 
Fig.  4.1.  For  large  minority  chemical  potentials,  the  critical  temperature  is  much  smaller, 
and  eventually  reaches  our  numerical  limit  (A^  ~  10“^t).  Before  this  limit  is  reached,  the 
flow  of  minority  pairing  is  stronger  than  majority  pairing  and  other  channels. 
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4.4  Summary 

We  have  performed  a  weak-coupling  FRG  study  of  a  population  imbalanced  fermion 
system  on  a  square  optical  lattice  in  two  dimensions.  At  the  weak  imbalance  region,  the 
competition  between  spin  fluctuations  and  intra-species  attractions  leads  to  a  stripe  den¬ 
sity  wave  phase,  rather  than  the  usual  uniform  SDW  phase.  As  the  imbalance  is  further 
increased,  p-wave  superfluid  phases  become  dominant  due  to  the  mismatch  of  the  FS.  Al¬ 
though  the  p-wave  pairing  superfluid  may  be  expected  in  dilute  density  limit,  the  com¬ 
petition  between  majority  and  minority  pairing  is  determined  by  their  FS  topology  and 
the  mediated  intra-species  pairing  interaction  which  is  renormalized  from  the  initial  bare 
interspecies  on-site  repulsive  interaction.  Both  the  stripe  density  wave  phase  and  triplet 
superfluid  phases  are  enhanced  by  nesting  of  the  majority  FS,  leading  to  much  higher  crit¬ 
ical  temperatures  than  that  found  for  the  imbalanced  Fermi  gas  without  lattice  [73,  74] 
at  weak  coupling  limit.  According  to  our  calculation,  the  critical  temperature  for  stripe 
density  wave  is  roughly  =  0,  =  O.Olt)  O.lSt  and  that  for  p-wave  superfluid 

is  Ac  =  O.OOOSt,  m  =  0.03f)  ss  O.OlGf,  where  the  hopping  amplitude  t  can  be  con¬ 

trolled  experimentally.  In  a  recent  experiment  [61],  it  is  reported  that  temperatures  around 
6%  of  the  Fermi  temperature  of  a  noninteracting  trapped  gas  have  been  achieved.  The 
symmetry  of  the  order  parameter  of  each  instability  can  be  probed  by  momentum-resolved 
spectroscopy  [84,  85].  These  techniques  provide  analogies  of  angle-resolved  photoemission 
spectroscopy  (ARPES)  used  in  condensed  matter  systems,  and  can  probe  anisotropic  sys¬ 
tems,  such  as  the  stripe  CDW  and  the  p-wave  pairing  that  we  predict  here. 
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Figure  4.1:  Phase  diagram  of  spin-imbalanced  fermion  mixture  with  majority  species  exactly 
at  half  filling  m  Q  (upper  panel)  and  slightly  away  from  half-filling  m  =  O.OOOSt  (lower 
panel).  The  system  goes  from  SDW  (black)  at  near  balance  to  a  stripe-CDW  (red)  as  the 
imbalance  increases.  The  insert  shows  the  strong  imbalance  region,  where  single  species 
p-wave  majority  (magenta)  or  minority  (blue)  pairing  is  the  dominant  instability. 
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Figure  4.2:  (a)  RG  flow  of  SDW^  (black)  and  CDW  (red)  channels  at  different  minority 
fillings,  where  =  0  (solid  line),  =  O.OOlt  (dashed  line),  and  =  O.lt  (dotted  line) 
with  majority  species  half- filled  (^^i-  =  0).  (b)  One  example  of  non-perfect  nesting  particle- 
hole  process  with  net  momentum  (27r,0).  (c)  The  symmetry  of  the  order  parameter  of 

majority  CDW  (yellow  circle)  and  minority  CDW  (cyan  square)  for  the  stripe-CDW  phase 
(/U  =  O.Olt).  A  conventional  s-wave  form  factor  (black  triangular  line)  is  also  shown.  Both 
species  behave  like  stripe  density  wave  along  the  diagonal  direction.  The  circles  on  the  FS 
are  FRG  patches. 


37 


52 


Figure  4.3:  (a)  RG  flow  of  majority  (magenta)  and  minority  (blue)  p-wave  BCS  channels 
at  three  different  minority  fillings,  where  =  0.5t  (solid  line),  =  0.8t  (dashed  line),  and 
fj-l  =  1.5t  (dotted  line)  with  majority  species  half-filled  (^-f-  =  0).  In  (b),  =  0.8t  (blue), 

the  renormalized  majority  pairing  channel  —k,  —k',  k')  couples  to  its  own  nesting 

channel  U-^{k,  —k,  —k  -|-  Q,  fc  +  Q).  In  (c),  m  =  1.5t,  the  half-filled  majority  species  can 
provide  more  phase  space  to  renormalize  minority  pairing  channel  (i.e,  q  can  be  anywhere 
on  that  FS  branch). 
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Chapter  5 

Quantum  Degenerate  Fermi-Fermi 
Mixture 

5.1  Introduction 

Experimental  realization  of  quantum  degenerate  Fermi-Fermi  mixtures  have  opened 
new  arena  for  the  study  of  quantum  many-body  phenomena  in  cold  atom  systems  [62, 
86,  87,  13,  88].  In  these  multi-component  systems,  many  different  types  of  fermionic  su¬ 
perfluids  have  been  proposed  and  studied  [89,  90].  For  imbalanced-population  mixtures, 
a  number  of  exotic  unconventional  pairing  states,  some  of  which  do  not  occur  in  nor¬ 
mal  condensed-matter  systems,  have  been  investigated  theoretically,  such  as  Fulde-Ferrell- 
Larkin-Ovchinnikov  [67,  66]  superfluidity  in  the  vicinity  of  phase  separation  [72],  interior 
gap  superfluidity  [69],  and  p-wave  pairing  [74,  9,  73].  However,  once  strong  Fermi  surface 
(FS)  nesting  is  present,  instead  of  superconducting  pairing,  density  wave  instabilities  domi- 
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nate  [9,  57]  and  may  exhibit  higher  angular  momentum  order.  Unconventional  density  waves 
(density  waves  with  non-zero  angular  momentum)  [15,  91,  92]  have  recently  been  proposed 
theoretically  for  fermionic  ultracold  atoms.  By  loading  fermionic  dipolar  atoms  [93]  and 
molecules  [94,  95,  96]  onto  optical  lattices,  unconventional  density- wave  instabilities  may 
arise  from  the  long-range  and  anisotropic  dipole-dipole  [97,  98,  99,  83,  100]  or  quadrupo- 
lar  [101]  interactions.  By  creating  a  Kagome  optical  lattice  [40],  a  bond  order  wave  can  be 
stabilized,  resulting  from  sublattice  interference  effects  [102],  in  the  presence  of  a  sufficiently 
large  nearest-neighbor  repulsive  term  [41,  103]. 

In  this  chapter,  we  study  density  wave  instabilities  of  a  doubly-degenerate  fermion 
mixture,  such  as  ®Li  and  [13,  104],  on  a  square  lattice  employing  a  functional  renormalization- 
group  (FRG)  method  [3,  5,  6,  51]  that  takes  into  account  inter-  and  intra-species  interactions 
on  an  equal  footing.  For  large  intra-species  repulsion,  the  behavior  is  reminiscent  of  a  single 
SU{2)  species.  When  both  species  fermions  are  at  half-filling,  and  in  the  presence  of  an 
inter-species  repulsion  sufficiently  larger  than  the  intra-species  repulsion,  the  ground  state 
consists  of  two  conventional  s-wave  number  density  waves,  here  denoted  as  charge-density- 
wave  (CDW)  in  analogy  with  number  density  waves  in  electron  systems.  As  one  species  is 
moved  away  from  half- filling,  a  dxy-wave  CDW  instability  becomes  dominant.  To  further 
elucidate  the  physics  behind  the  d^y-CDW  formation,  we  develop  a  simple  four-patch  model 
with  intra-  and  inter-species  Umklapp  couplings  that  provides  an  analytical  understanding 
of  the  competition  between  s-  and  dxy-CDWs.  A  density  imbalance  between  the  two  species 
of  fermions  in  the  vicinity  of  half-filling  leads  to  different  phase  spaces  for  the  different  inter¬ 
species  Umklapp  couplings.  Using  reasoning  similar  to  Shankar’s  phase  space  argument  for 
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determining  one- loop  RG  flow  equations  [3],  we  explain  how  the  phase-space  discrepancy 
for  the  inter-species  Umklapp  couplings  leads  to  opposite  RG  flows  for  the  different  intra¬ 
species  Umklapp  couplings,  triggering  the  dxy-CDW  instability.  By  varying  the  densities  of 
the  two  species  around  half-filling,  we  also  study  the  range  of  fillings  for  which  this  regime 
of  unconventional  dxy-CDW  can  be  realized. 

5.2  Model  and  Formalism 

We  start  with  a  simple  Hamiltonian  describing  a  fermion-fermion  mixture  on  a 
square  lattice,  H  =  +  Hf  -|-  Hint,  where 

+  H-C.^  -  nai,  (5.1) 

{ij},a  i 

Hint  —  ^  ^  Ucc'^ci^'^ci^  “1“  (^•^) 

i 

where  a  =  c//  stands  for  the  c//-fermions,  a  is  the  spin  index,  (ij)  represents  nearest- 
neighbor  pairs  of  sites,  nai  =  chemical  potential  of 

the  a-fermions.  For  simplicity,  we  set  nearest  neighbor  tunneling  tc  =  tf  =  t.  We  consider 
the  limit  of  weak  repulsive  interactions  and  employ  a  standard  FRG  method  [3,  5,  6,  51]  to 
obtain  the  low-energy  behavior  for  this  system,  expressing  the  quartic  terms  in  the  resulting 
effective  Hamiltonian  in  momentum  space  in  the  form 

gabab{kl,k2,k3,i)'ll;l^{k3)'lpli^{ki  +  k2  -  k3)'lpb0{k2)lpaa{kl)  ,  (5.3) 

,  where  V’aa  is  the  creation  operator  of  a-fermions  in  momentum  space,  k  =  {kx,ky),  spin 
indices  a,  j3  =t,  i,  species  indices  a,b  =  c,  f  and  £  =  ln(8t/A),  with  A  the  UV  energy  cut-off 
and  8t  the  full  energy  bandwidth. 


41 


56 


5.3  Results 

With  fixed  parameters  (in  units  of  t),  fif  =  0  (half- filling)  and  Us  =  Ucc  >  0, 
the  phase  diagram  parameterized  by  and  the  dimensionless  ratio  of  intra-  and  inter¬ 
species  interaction  Ud/Us  is  illustrated  in  Fig.  5.1.  For  small  inter-species  interactions,  the 
species  at  half-filling  (/-fermions)  has  an  SDW  instability.  With  increasing  Ud/Us,  if  both 
species  are  at  half-filling,  an  s-wave  CDW  emerges,  where  the  order  parameters  for  the 
two  species  have  a  7r-phase  difference,  referred  as  s±-CDW  here.  This  is  analogous  to  the 
s±-wave  pairing  in  iron-pnictide  superconductors  [33,  14],  but  in  the  particle- hole  channel. 
By  relabeling  the  square  lattice  as  a  bipartite  lattice  with  A,B  sub-lattices,  the  real-space 
picture  of  the  s±-CDW  phase  is  that  /(c)-fermions  only  occupy  A{B)  sub-lattice  sites  to 
avoid  the  strong  inter-species  repulsion. 

Keeping  the  /-fermions  at  half-filling,  we  find  phase  transitions  from  s±-CDW 
to  dxy-CUW  by  increasing  l/rd,  and  from  SDW  to  d^y-CDW  by  enhancing  Ud/Us-  In 
Fig.  5.2(left),  we  show  the  FS  of  the  two  species  for  fif  =  0  and  fic  =  —0.1  and  indicate  the 
FS  patches  used  in  our  numerical  RG  implementation.  We  also  show,  in  the  same  figure,  the 
corresponding  form  factors  of  dxy-CDW  for  Ud/Us  =  2.5,  obtained  from  decomposing  the 
CDW  coupling  into  eigenfunctions  and  plotting  the  dominant  one  close  to  the  instability. 
The  larger  magnitude  of  the  form  factor  for  /-fermions  indicates  that  the  (i2;y-instability  is 
mainly  driven  by  the  species  at  half-filling,  with  ordering  of  the  c-fermions  resulting  from 
a  proximity  effect.  As  we  increase  the  density  imbalance,  the  dxy-CDW  phase  requires 
larger  Ud/Us  to  dominate  over  SDW.  The  real-space  picture  of  dxy-CDW  is  sketched  in 
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Figure  5.1:  The  phase  diagram  is  parameterized  by  the  chemical  potential  of  c-fermions, 
/ic,  and  the  ratio  of  inter-  and  intra-species  interactions,  Ud/Us. 


Fig.  5.2(right),  where  the  crosses  indicate  higher  densities  along  next-nearest-neighbor  bonds 
of  alternating  plaquettes,  OdCDw(’’)  =  Y.a-,i,j=±i  2/)«a(a^  +  b  2/  +  j)  [92]. 

5.3.1  4-patch  toy  model 

To  understand  the  physics  leading  to  the  development  of  the  dxy-CDW,  we  first  ex¬ 
plore  the  competition  between  the  s-CDW  and  the  dxy-CDW  of  a  single  species  (/-fermions) 
at  half-filling.  By  decoupling  the  quartic  interaction  terms  in  the  form  ^ 
with  Ok  being  a  term  bilinear  in  the  fermion  fields  and  corresponding  to  the  order  parameter 
of  CDW,  VcDw(fc)P)  —  gss{f^,P,P  +  Q,  fc  +  Q)  exhibits  a  negative  eigenvalue  when  CDW 
dominates  over  the  SDW  [33,  14].  In  this  case,  for  simplicity,  we  can  divide  the  FS  into  four 
patches  around  kp  =  (7r/2,7r/2),  (— 7r/2,7r/2)  (— 7r/2,  — 7r/2)  and  (7r/2,— 7r/2)  and  consider 
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Figure  5.2:  (Left)  dxy  order  parameter  for  /-  and  c-fermions.  (Right)  Real-space  particle 
density  of  the  dxy-CDW  state  for  one  species  of  fermions. 


only  the  scatterings  occurring  on  these  pieces  of  the  FS  (Fig.  5.3a).  Under  the  dihedral  sym¬ 
metry  of  a  square  lattice,  three  independent  Umklapp  couplings  associated  with  momentum 
transfer  —  ki  =  Q  =  (vr,  vr)  can  be  defined:  gi  =  (3,  3, 1, 1),  g2  =  <7^(1, 4,  2,  3)  and 

93  =  fl'fffr(l,  3,  3, 1).  The  matrix  of  Vcdw  becomes 

/  \ 


Vcdw  — 


91 

92 

93 

92 

92 

91 

92 

93 

93 

92 

91 

92 

92 

93 

92 

91 

(5.4) 


with  eigenvectors  =  (1,1, 1,1),  =  (1)~1)1)~1))  ^pi  =  (IjO,  — 1,0)  and  = 

(0, 1,0,  —1),  and  corresponding  eigenvalues  Eg  =  gi  +  2g2  +  93,  Ed  =  gi  —  2g2  +  93  and 
Upi/2  =91  —93-  Thus,  once  the  CDW  instability  is  triggered,  the  competition  between  d- 
and  s-wave  channels  is  clear:  dxy-CDW  (s-CDW)  has  the  most  negative  eigenvalue  and  is 
therefore  dominant  when  g2  >  0  (772  <  0). 

We  now  introduce  the  second  species  of  fermions  (c-fermions),  with  hlling  slightly 
less  than  half  and  with  the  FS  also  divided  into  four  patches  (Fig.  5.3).  By  introducing 
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Figure  5.3:  The  sketches  of  two  inter-species  couplings,  =  (7cfcf(3, 3, 1, 1)  (left)  and 
92  ~  ffcrcf  (4, 1,  2,  3)  (right),  for  two  different  fillings  of  c-fermions.  In  (b),  the  deviation 
of  the  FS  for  gc  =  —0.1  from  the  one  at  half-filling  is  intentionally  enlarged  for  better 
visualization. 


an  inter-species  interaction  and  dehning  the  Umklapp  couplings  <7^^  =  5cfcf(3,3, 1, 1),  §2^  = 
9cfcf(4, 1,2,3)  and  =  (7cfcf(l, 3, 3, 1),  the  one-loop  RG  equations  for  gi  and  g2  are  given 
by 


—  Aljkgjgk  +  +^122  (^2^)  )  (5-5) 

n=l 

=  A2jk9j9k  +  21321251^52^  +  2^22352^53^,  (5-6) 
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where  Aijk  and  Bijk  are  the  kernels  of  the  RG  equations  for  intra-  and  inter-species  cou¬ 
plings,  respectively,  and  i,j, /c  represent  coupling  indices.  Aijk  contains  contributions  from 
all  one- loop  diagrams  (Figs.  2.2b  to  2.2f).  Bijk  corresponds  to  a  fermionic  bubble  (Fig.  2. 2d) 
for  the  c- fermions,  obtained  when  two  inter-species  couplings  are  contracted  to  generate  a 
correction  to  the  intra-species  /-fermion  coupling.  The  BijkS,  are  therefore  always  negative 
and  correspond  to  an  effective  attractive  interaction  for  the  /-fermions,  mediated  by  the 
c-fermions.  Note  that  when  the  bare  inter-species  interaction  is  weak,  Ud/Us  <C  1,  and  the 
Bijkgfg"^  terms  can  be  neglected.  In  this  case,  the  Aijkgjgk  intra-species  terms  eventually 
drive  gi  and  g2  to  large  positive  values,  leading  to  an  SDW  instability. 

When  both  species  are  at  half-filling,  the  two  FS  overlap  and  the  three  inter¬ 
species  Umklapp  couplings  g^^,  g^^)  exhibit  equivalent  phase  space;  the  same  applies 
to  the  intra-species  Umklapp  couplings.  In  this  circumstance,  increasing  the  bare  value  of 
the  inter-species  interaction  in  Eq.  (5.5)  and  (5.6)  eventually  leads  both  gi  and  g2  to  flow 
to  negative  values  and  gives  rise  the  s-CDW  instability.  When  one  species  is  away  from 
half-filling,  the  four  patches  on  its  rounded-square  FS  are  slightly  shifted  from  the  ones 
at  half-filling.  In  Fig.  5.3b,  we  illustrate  the  two  FS  and  corresponding  momentum  shifts 
between  the  patches.  As  shown  in  the  left  panel  of  Fig.  5.3b,  for  fermions  on  the  FS,  the 
net  momentum  transfer  in  the  inter-species  process  gf  is  no  longer  equal  to  a  reciprocal 
lattice  vector  but  has  a  small  shortage  corresponding  to  the  shift  in  the  position  of  the 
Fermi  patches  for  the  c-fermions.  This  is  also  true  for  the  g^  processes.  As  high-energy 
modes  are  eliminated  during  the  RG  transformation,  these  two  couplings  will  have  a  reduced 
phase-space  in  which  they  can  occur  and  are  not  allowed  when  only  fermions  on  the  FS 
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remain.  However,  the  shortage  in  the  momentum  transfer  for  scattering  c-fermions  across 
two  parallel  patches  (momentum  transfer  Q  —  S)  can  be  compensated  in  §2^  by  having  the 
/-fermions  scatter  by  Q-|-<5.  As  sketched  in  Fig.  5.3b  (right  panel),  this  scattering  is  allowed 
by  momentum  and  energy  conservation  due  to  the  nesting  of  the  FS  (flat  parallel  patches 
in  our  simple  model).  Thus,  after  moving  one  species  away  from  half- filling,  following 
Shankar’s  phase-space  argument  [3],  particle-hole  diagrams  involving  §2^  have  an  extended 
phase  space  in  which  they  are  non-zero,  while  ones  involving  and  can  be  ignored 
to  leading  order  in  dA.  As  a  consequence,  the  RG  equations,  Eq.  (5.5)  and  (5.6),  can  be 
rewritten  as  dgi/di  =  Aijkgjgk  +  2H122  dg2/di  =  A2jk9jgk-,  and  g2  receives  no 

contribution  from  inter-species  terms.  The  inter-species  interaction  term  drives  gi  to  a 
negative  value  (effective  mediated  attraction),  but  g2,  which  is  renormalized  only  by  intra¬ 
species  repulsion,  still  flows  to  positive  values.  The  flow  to  negative  values  leads  to  CDW 
instability,  and  the  asymmetry  in  the  Umklapp  components  means  that  the  CDW  instability 
has  dxy-wave  order  parameter  {gi  <  0  and  g2  >0). 

5.3.2  Grand  FRG  flow 

Going  back  to  the  full  FRG  calculation  with  2x36  patches,  we  confirm  the  simple 
mechanism  demonstrated  with  the  four-patch  model  by  identifying,  among  the  RG  flows 
for  all  the  couplings,  the  ones  that  are  most  divergent  and  looking  at  their  behavior.  In 
Fig.  5.4,  the  RG  flows  of  the  most  dominant  intra-species  couplings,  Uss-,  are  plotted  for  two 
different  cases:  c-fermions  at  half-filling  (//c  =  0)  and  away  from  half- filling  {pLc  =  —0.1). 
When  both  species  are  at  half-filling,  the  two  couplings  are  equal  to  each  other  and  flow 
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Figure  5.4:  The  flows  of  dominant  nesting  couplings  in  the  RG  procedure,  g'fjg,  are  illustrated 
for  /Xc  =  0  (dashed  lines)  -  note  that  these  two  lines  are  on  top  of  each  other  indicating  that 
the  two  couplings  have  the  same  magnitude  in  this  case;  and  /ic  =  —0.1  (solid  lines).  The 
three  couplings  are  illustrated  in  the  insets,  and  their  flows  with  matched  color. 


to  large  negative  values  (dashed  and  solid  blue  lines),  leading  to  the  s-CDW  instability. 
However,  as  expected  from  the  four-patch  analysis,  once  the  c-fermions  are  away  from  half¬ 
filling  (/ic  =  —0.1),  the  two  couplings  are  renormalized  in  opposite  ways  (dashed  and  solid 
orange  lines),  resulting  in  dxy-CDW .  The  population  imbalance  between  the  two  fermion 
species  in  the  vicinity  of  FS  nesting  provides  the  two  FS  with  displaced  parallel  portions, 
which  is  the  key  ingredient  for  the  mechanism  we  propose  for  creating  a  d^y-CDW  with 
purely  repulsive  interactions. 

In  our  numerical  calculation  we  obtain  the  RG  flows  of  all  interaction  vertices 
with  momenta  lying  on  the  Fermi  surfaces.  Although  we  can  explain  the  dxy-CDV^  phase 
using  a  simple  four-patch  model  and  analytical  arguments  involving  RG  flows  of  just  a  few 
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Figure  5.5:  Snapshots  of  interaction  vertices  during  RG  flow  for  /-fermions  at  half- filling 
{fif  =  0).  (a)  gcfcf(R  =  and  (b)-(d)  gss{ii  =  Other  parameters  are  set 

to  fic  =  0,  Uci/Us  =  2.0  for  (a)  and  (b),  where  the  s±-CDW  instability  occurs.  In  (c), 
/ic  =  O.lt  and  I7cf/b^fr  =  2.0,  this  is  in  the  regime  where  unconventional  dxy-wave  CDW 
dominates,  (d)  gc  =  O.lt,  Ud/Us  =  0.5,  the  SDW  will  be  the  instability  if  inter-species 
interaction  is  weak.  Intra-species  interactions  are  set  to  Ucc  =  =  l.Ot  in  all  cases  shown 

here. 


Umklapp  couplings,  it  is  instructive  to  see  how  some  of  the  vertex  functions  renormalize. 
In  Fig.  5.5,  we  show  a  snapshot  at  a  given  “RG-time”  £  of  a  few  coupling  functions.  Since 
each  vertex  gababih,  ^2,^3^^)  is  a  function  of  three  free  patch-indices,  we  fix  one  of  them, 
^2  =  1,  where  the  positions  of  each  patch  on  the  two  Fermi  surfaces  are  illustrated  in  Fig. lb 
of  the  main  text.  For  /-fermions  at  half-filling  (/ij  =  0)  and  fixing  Ucc  =  Ug  =  t,  we 
show  couplings  for  different  /ic  and  Ud/Us  corresponding  to  the  s±-GDW,  d^y-CDW,  and 
SDW  phases.  Fig.  5.5a  and  Fig.  5.5b  show  inter-species  coupling  gddih  =  1G2G3,^)  and 
/-fermions  intra-species  coupling  gm{ii  =  ^,i2,i3,^),  respectively,  for  fic  =  ^  and  Ud/Us  = 
2.0,  corresponding  to  a  point  in  the  phase  diagram  where  s±-GDW  occurs  (see  Fig.  la 
in  the  main  text).  Moving  the  c- fermions  away  from  half- filling  leads  to  a  region  of  dxy- 
CDW.  Fig.  5.5c  shows  the  intra-species  coupling  gssih  =  Ij  *2G3,  f*)  for  this  case  {fic  =  O.lt, 
Ud/Ufi  =  2.0).  Decreasing  the  inter-species  coupling  from  this  point  so  that  I/fj  >  Ud  leads 
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to  the  SDW  phase.  Intra-species  coupling  ^2,  ^3,  for  this  case  is  shown  in  Fig.  5.5d 

(^c  =  O.lt,  C/cf/C/ff  =  0.5). 

It  is  easy  to  see  from  the  panels  in  Fig.  5.5  that  flows  to  strong  coupling  under 
RG  are  related  to  Fermi  surface  nesting  due  to  commensurate  filling.  There  are  two  types 
of  such  couplings.  The  first  type  contains  couplings  of  the  form  5frfr(l,  ^2,  ^2  +  Q),  which 
correspond  to  the  four  pieces  of  diagonal  lines  (dark  red)  in  Fig.  5.5d.  These  couplings 
account  for  the  nesting  between  ki  and  fcs,  and  represent  the  SDW  channel  in  Eq.  (2.21). 
The  second  type  contains  (1,  ^2,  27)  couplings,  which  make  up  the  vertical  straight  line 
at  is  =  27  in  all  four  panels  of  Fig.  5.5  (blue  line  in  Fig.  5.5b,  in  particular),  and  are 
responsible  for  the  CDW  channel.  One  can  also  see  some  sub-dominant  vertices  in  these 
figures.  The  pairing  vertex,  5frfr(ii,  ^2  =  — T,  is)  corresponds  to  the  horizontal  line  at  fi  =  19. 
The  major  diagonal  line  at  45°,  going  from  the  lower  left  to  the  upper  right  corners  of  the 
panels,  corresponds  to  the  backward  scattering,  =  ^2)-  The  forward  scattering, 

fl'fffr(fi,  ^2)  fs  =  *1))  is  the  vertical  straight  line  located  at  is  =  1.  Finally,  there  are  four 
small  squares,  which  are  enhanced  due  to  scattering  of  a  particle  from  one  patch  to  another 
belonging  to  the  same  flat  side  of  the  nested  Fermi  surface. 

Starting  from  the  s±-CDW  phase  (Fig.  5.5a  and  5.5b),  the  leading  divergent  cou¬ 
plings  are  of  the  second  kind,  (1,  i2,  27),  and  flow  to  negative  values  (vertical  line  along 
is  =  27  in  Fig.  5.5b)  as  we  mention  in  the  main  text.  The  points  along  this  vertical  straight 
line  of  most  divergent  couplings  have  the  same  magnitude  for  all  values  of  i2,  indicating  an 
isotropic  s-wave  symmetry.  Furthermore,  the  inter-species  interaction,  5rcfcf(l,  i2)  27),  flows 
to  strong  positive  values  (5.5a),  again  with  the  same  value  for  all  ii  patch  indices.  This  sign 
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difference  between  and  ^fcfcf  indicates  that  the  s-wave  order  parameter  has  a  7r-phase 
between  the  /-  and  c-fermions,  and  thus  this  corresponds  to  an  s±-CDW  instability. 

In  Fig.  5.5d,  intra-species  couplings  are  almost  unaffected  by  weak  inter-species 
couplings,  and  the  strongest  flow  is  in  the  SDW  channel,  which  is  represented  by  the  four 
separate  pieces  of  diagonal  red  lines.  Finally,  in  Fig.  5.5c,  the  couplings  show  that  the 
nesting  process,  (1,  Z2,  27),  has  alternating  signs  (red  and  blue  in  the  vertical  straight 
line  at  is  =  27).  The  sign  alternation  in  the  CDW  form  factor  with  four  nodes,  at  (±7r,  0) 
and  (0,  ±7r),  corresponds  to  dxy-wave  symmetry,  instead  of  the  more  usual  isotropic  s-wave 
symmetry  seen  in  Fig.  5.5b.  The  physics  behind  the  emergence  of  this  sign  change  and 
dj^j^-symmetry  is  explained  in  main  text. 

5.3.3  Critical  Temperature 

During  the  FRG  calculation,  the  UV  cutoff  =  Aoe“^  is  reduced  as  the  RG  scale 
I  increases.  The  scale  I  =  Ic  at  which  an  instability  occurs  (divergent  flow)  provides  an 
estimate  for  the  critical  energy  (and  temperature)  scale  for  the  transition[3].  We  first  use 
a  small  initial  bare  coupling,  Ucc  =  Us  =  0.2t,  and  estimate  the  critical  temperature.  The 
phase  diagram  for  this  case  is  shown  in  Fig.  lb  of  the  main  text.  In  Fig.  5.6a,  we  show 
the  critical  temperature  from  our  FRG  calculation  as  a  function  of  the  ratio  Ucf/Us,  for 
Ucc  =  Us  =  0.2t,  for  a  fixed  value  of  g,c  =  0.2t.  This  corresponds  to  a  vertical  cut  of 
the  phase  diagram  shown  in  Fig.  lb  of  the  main  text.  There  are  two  competing  orders, 
SDW  and  dxy-CDW,  and  their  corresponding  Tc  in  units  of  the  Fermi  temperature  Tp. 
The  critical  temperatures  found  for  this  case  are  small,  less  than  lO^^Tp.  However  the 
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Figure  5.6:  Critical  temperatures  in  units  of  the  Fermi  temperature  Tp  for  the  dominant 
instabilities  SDW  and  d^j^-CDW  as  a  function  of  the  interaction  ratio  Ud/Ug,  with  /ij  =  0 
and  /ic  =  0.2t. 


critical  temperature  is  significantly  larger  for  larger  bare  interaction  strengths,  as  shown 
in  Fig.  5.6b  for  Ucc  =  Ug  =  t,  where  Tc  of  the  dxy-CDW  is  more  than  10%  of  Tp.  Both 
figures  (Fig.  5.6a  and  b)  show  that  the  Tc  of  SDW  decreases  as  the  ratio  of  initial  bare 
interactions  Ud/Ug  increases  because  it  is  stabilized  by  the  on-site  repulsion  for  the  species 
at  half- filling  (/-fermions)  and  larger  on-site  inter-species  repulsion  ?7cf  acts  against  it. 
As  the  other  competing  order  dxy-CTTN  starts  to  dominate,  its  Tc  increases  as  the  ratio 
increases.  Larger  initial  on-site  inter-species  repulsion  Ud  favors  the  dxy-CDW  instability. 
While  the  FRG  approach  only  applies  to  weak-couplings,  that  is,  bare  coupling  strengths 
small  compared  to  the  full  energy  bandwidth  of  the  problem  (of  order  8t),  the  ratio  of 
Ud/Ug  can  be  tuned  to  be  very  large,  enhancing  the  value  of  Tc  for  dxy-CTTN.  In  a  cold 
atom  experiment,  the  /-fermions  could  be  taken  to  be  ®Li  or  for  example,  and  the 
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intra-species  interaction  Uq  can  be  tuned  to  close  to  zero  via  a  Feshbach  resonance  with 
remaining  positive  and  smaller  than  8t. 

5.3.4  Robustness  of  the  (Ixy-CDW  phase 

As  the  calculation  of  Tc  indicates,  a  sufficiently  large  inter-species  interaction  (t/cf) 
compared  to  the  intra-species  repulsion  of  the  species  at  half-filling  ([%)  is  needed  to  create 
the  dxy-CDW  phase  with  a  realistic  value  for  Tc.  However,  it  is  not  necessary  to  have  the 
two  intra-species  interactions.  Us  and  Ucc,  to  be  equal.  If  they  are  different,  the  phase 
diagram  as  a  function  of  the  ratio  Ud/Ug  remains  qualitatively  unchanged,  with  no  strong 
dependence  on  the  value  of  Ucc-  We  show  the  phase  diagram  for  Ucc  =  0  in  Fig.  5.7(a). 
Comparing  with  Fig.  lb  in  the  main  text,  we  see  that  it  is  qualitatively  the  same,  with 
the  phase  boundaries  only  slightly  shifted,  and  therefore  the  exotic  dxy-CDW  phase  is  not 
sensitive  to  the  c-fermion  intra-species  interaction. 

Experimental  candidates  for  realizing  this  mixture,  such  as  ®Li-^^K  mixture,  have 
components  with  different  values  for  their  physical  mass.  The  qualitative  features  of  the 
phase  diagram  indeed  do  not  require  tf  =  tc-  If  the  c-fermion,  which  is  the  species  not  at 
half-filling,  is  lighter  than  the  /-fermion,  the  screening  effect  from  c-fermions  becomes  even 
more  effective  than  in  the  equal  mass  case,  and  the  dxy-CDW  phase  is  further  enhanced. 
If  the  c-fermions  are  heavier,  the  screening  effect  becomes  less  effective  and,  for  c-fermions 
much  heavier  than  /-fermions,  the  dxy-CDW  will  eventually  be  completely  suppressed. 
Therefore,  for  a  ®Li-^^K  mixture,  having  the  atoms  be  the  /-fermions  (half-filled)  is 
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more  favorable  for  the  emergence  of  d^^y-CDW  order.  These  features  provide  more  flexibility 
for  the  experimental  realization  of  the  dxy-CTnW  phase  we  predict. 

The  effective  attractive  interaction  between  /-fermions,  mediated  by  the  c-fermions, 
can  also  drive  a  BCS  pairing  instability  if  the  nesting  of  the  /-fermions  FS  is  destroyed. 
To  study  the  competition  between  pairing  (SC)  and  density-wave  instabilities,  we  explored 
the  phase  diagram  parameterized  by  the  chemical  potentials  of  the  two  species,  shown  in 
Fig.  5.7(b)  for  Ucc  =  t/fr  =  t/cf/2.5.  The  d^jy-CDW  occurs  as  an  intermediate  phase  between 
two  limiting  behaviors:  s±-CDW,  when  both  species  are  at  half-filling,  and  SDW,  when 
one  species  is  at  half-filling  and  the  other  is  far  away  from  half-filling.  Importantly,  the  dxy 
phase  persists  in  a  reasonably  wide  range  of  fillings  of  the  minority  species. 

The  scale  (.c  at  which  the  divergence  of  the  couplings  occurs  provides  an  estimate 
for  the  critical  energy  (and  temperature)  scale  for  the  transition.  For  the  dxy-CDS^  tran¬ 
sition  we  find  that  for  =  0.2t,  the  optimal  Tc  is  about  0.01  Tp.  For  Up  =  t,  Tc  is 
considerably  higher,  ranging  from  0. 1-0.4  Tp  for  different  values  of  the  Ucf/Up  ratio.  Ex¬ 
periments  with  ®Li-‘^°K  mixtures  [104]  have  achieved  Tu  =  O.SdTp  and  Tk  =  OAOTp.  The 
values  we  find  for  Tc  using  FRG  are  only  rough  estimates,  but  they  are  of  the  same  order 
of  magnitude  as  these  experimental  values. 

5.4  Summary 

We  point  out  that  a  sufficiently  large  inter-species  interaction  compared  to  the 
intra-species  repulsion  of  the  species  at  half-filling  Up  is  needed  to  create  the  dxy-CDW  we 
propose  here.  However,  it  is  not  necessary  to  have  Up  =  Ucc-  If  they  are  different,  the  phase 
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Figure  5.7:  (a)  The  phase  diagram  for  non-interacting  c-fermions  {Ucc  =  0),  parameterized 
by  the  chemical  potential  ^c-,  and  the  ratio  of  inter-  and  intra-species  interactions,  Ud/Ug. 
(b)  The  phase  diagram  parameterized  by  fic  and  Hf.  Note  the  large  difference  in  scales 
between  fic  and  /rj. 


diagram  as  a  function  of  the  ratio  Ud/Us  remains  qualitatively  unchanged,  with  no  strong 
dependence  on  the  value  of  C4c-  The  qualitative  features  of  the  phase  diagram  also  do 
not  require  tf  =  tc-  Our  predicted  phase  diagram  should  be  experimentally  realizable  with 
recently  achieved  quantum  degenerate  Li-Yb  [88]  or  Li-K  mixtures  [13]  on  a  square  optical 
lattice.  Taking  the  ®Li-^^K  mixture,  for  example,  with  as  the  /-fermions  at  half-hlling, 
one  can  tune  t/g  with  a  magnetic  Feshbach  resonance  (Uq  ~  0  at  around  210  Gauss).  At 
magnetic  fields  in  this  range,  the  ®Li  -  ®Li  interaction  Ucc  is  small  and  repulsive,  as  well 
as  the  interspecies  interaction  Ud-  One  can  therefore  use  the  Feshbach  resonance  to 
tune  the  Ucf/Us  ratio  to  large  values,  as  required  for  our  predicted  d-CDW  phase  to  occur. 
One  may  also  combine  magnetic  held  with  conhnement-induced  resonance  [105,  106,  107]. 
In  order  to  avoid  breaking  SU(2)  symmetry,  optical  Feshbach  resonances  [108,  109]  may 
also  be  explored.  The  all  optical  resonance  has  been  used  in  fermionic  systems  [110,  111], 
including  the  mixture  [13]. 
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Identification  of  order  parameters  with  nonzero  angular- momentum  dependence  re¬ 
quires  measurement  of  the  relative  phase  between  different  portions  of  the  FS.  Several  meth¬ 
ods  have  been  proposed  to  perform  such  phase-sensitive  measurements  [112,  84,  113,  114], 
including  a  pump-probe  scheme  [115]  and  through  noise  correlations [114],  The  dxy-CDW 
order  parameter  symmetry  may  also  be  detected  via  momentum-resolved  spectroscopy  [84, 
85]. 
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Chapter  6 

Dipolar  Fermions  on  Square 
Lattice  of  Bilayer  System 

6.1  Introduction 

The  rapid  progress  in  trapping  and  cooling  of  ultracold  magnetic  atoms  [93]  or 
polar  diatomic  molecules  [94]  provides  a  new  avenue  for  studying  strongly  correlated  effects 
and  many-body  physics  [116].  The  magnetic  dipole  atoms,  dysprosium,  has  been  cooled  to 
low  temperature  and  reach  quantum  degeneracy  [93].  The  potassium-rubidium  molecules 
have  been  achieved  [94,  95],  and  importantly  confined  to  2D  layers  using  optical  lattice  [117]. 
The  advantage  of  loading  molecules  onto  optical  lattice  is  that  makes  system  more  stable 
from  quantum  chemical  reaction  loss  due  to  attractive  interaction  [118,  117].  On  the  theory 
side,  the  anisotropic  and  long-range  nature  of  dipole-dipole  force  might  leads  to  more  exotic 
phases  or  collective  modes  in  multilayer  stack  at  low  temperature  [119,  120]  ,  including  stripe 
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density  wave  [121,  122],  bound  dimer  state  [123],  inter- layer  superfluidity  [124,  125,  126], 
and  maximally  entangle  state  [127]. 

Recently,  unconventional  density  wave  has  attracted  great  attention,  especially 
bond-order  wave  [15,  102,  83].  These  density  waves  usually  are  found  only  in  a  nar¬ 
row  parameter  regime,  where  the  frustration  occurs  for  usual  s-wave  charge  density  wave. 
For  example,  the  bond-order  p-wave  only  exists  in  a  small  regime  between  charge  density 
wave(CDW)  and  spin  density  wave  in  one-dimensional  extended  Hubbard  model  [24]  due 
to  strong  competition  between  on-site  and  nearest  neighbor  repulsive  interactions.  Despite 
their  ubiquity  and  potential  importance,  their  physical  origins  and  behaviors  are  still  under 
debate.  Several  mechanism  have  been  proposed  and  studied  to  discover  unconventional 
density  wave.  Both  charge-  and  spin  bond-order- wave  are  found  in  a  quantum-degenerate 
Fermions  on  kagome  lattice  [102,  103,  41],  where  the  sub-lattice  interference  effect  takes 
place.  The  other  proposal  is  using  density  imbalanced  Fermi-Fermi  mixture  [9],  in  which 
phase  space  discrepancy  leads  to  dxy-wave  charge  density  wave  on  square  lattice.  Finally, 
fermions  with  dipole  [83,  100]  or  quadruple  [101,  128]  nature  on  a  2D  square  optical  lat¬ 
tice,  the  unconventional  density  wave  is  found  for  both  single-  and  two-component  system, 
where  the  long-range  anisotropic  nature  of  dipole-dipole  interactions  is  the  key  ingredient. 
For  example,  half-filled  single  component  dipole  fermions  on  a  2D  square  lattice  form  a 
bond-order  solid  when  the  polar  angle  is  tilted  between  two  critical  values  [83]. 

In  this  letter,  we  use  functional  renormalization  group  (FRG)  to  study  single¬ 
component  dipolar  fermions  of  bilayer  with  presence  of  square  lattice.  This  method  has  been 
wildly  used  to  study  strongly  correlated  election  system,  such  as  Hubbard  model  [8,  6,  4], 
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and  cold  atoms  system  under  various  optical  lattice  geometry  [103,  129]  and  different  types  of 
interactions  [83,  100,  101].  For  weak  couplings,  density  wave  state  relies  on  commensurate 
filling,  which  provides  the  most  important  feature  -  nested  Fermi  surface(FS)  -  to  form 
modulations  of  particle  wave  function  in  real  space.  For  dipole  fermions  in  two  dimensional 
bilayer  geometry,  the  tilting  angle  can  be  controlled  by  applying  uniform  electric  or  magnetic 
fields.  Also,  the  layer  separation  can  be  tuned  by  using  two  counter  propagating  lasers  [116]. 
Under  different  tilting  angle  and  layer  separation,  the  anisotropic  and  long-range  nature 
of  dipole-dipole  interaction  within  the  layer  and  between  the  layers  play  a  key  role  in 
determining  quantum  phases  in  low  temperature.  Interestingly,  the  interplay  between  intra- 
and  inter-layer  interactions  give  the  exotic  unconventional  density  wave  phases.  We  discover 
the  exotic  bond  order  wave,  which  is  out-of-phase  between  layers.  Also,  this  unconventional 
charge  density  wave  lives  in  wilder  range  in  phase  diagram  than  one  in  monolayer  square 
lattice. 


6.2  Model  and  Formalism 


We  model  this  single-species  dipolar  Fermions,  with  creation  operator  c^,  on  a 
quasi- two-dimensional  bilayer  square  lattice  with  lattice  constant  and  layer  separation 
Iz,  saying 


%=  -  t  {cl^Cja  h.c.)  -  tz  y~l(4iC»2  +  h.C.) 

(ij)a  i 

M  ^  ^  '^ia  “1“  2  ^  ^  ; 

ia 


(6.1) 
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where  {ij)  representing  nearest  neighbor  sites,  layer-index  is  a,  ;0  =  1,  2,  and  density  operator 
is  defined  as  riia  =  c|^CjQ,.  We  assume  all  dipoles  are  aligned  in  the  same  direction  d  = 
(i(sin0£)  cos  (/>D,  sin sin  (/)£),  cos  0d)  by  applying  uniform  electric  or  magnetic  field.  Under 
this  assumption,  the  dipole-dipole  interaction  Vapifij)  can  be  determined  by  the  two-particle 
Wannier  basis,  and  resulted  in  anisotropic  form 


bl2(21) 


V22{nj)  =  Vd 


1  -  3(fy  •  d) 
ij 


l-3[(r-;,±U).d]7H+/2) 
"  (r2.  + 


(6.2) 

(6.3) 


for  intra-  and  inter- layer,  where  fij  =fi  —  fj  is  a  direction  on  2D  plane  between  two  dipoles. 


—  Ui(«£®)  - Viiia-LX+aiy)  ...U2(«i*) 

'““Ui(®ry)  ■■■  ^.2(0)  ■■■ 


Figure  6.1:  The  solid  (dashed)  lines  are  magnitudes,  in  terms  of  Vd,  of  intra-layer  Vu 
(inter- layer  V12)  dipole-dipole  interaction  under  different  polar  angle  0£).  The  azimuthal 
angle  of  dipoles  is  set  to  (/>£)  =  0°  in  (a),  and  (/>_d  =  45°  in  (b),  where  layer  separation 
is  Iz  =  aL-  Vii{aLx)  (red),  Vii{aLy)  (yellow),  and  Vi2(0)  (green)  are  nearest  neighbors. 
Vii{aLX+aLy)  (blue),  Vi2{aLx)  (magenta),  and  Vi2{aLy)  (cyan)  are  next-nearest  neighbors. 
The  two  critical  angles  (black  dotted  lines)  are  9ci  ~  35°  and  6c2  ~  55° . 
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In  monolayer  system,  the  low-temperature  phase  is  mainly  driven  by  anisotropic 
intra-layer  interaction.  From  Fig.  6.1(a),  we  can  easily  see  that  leading  intra-layer  interac¬ 
tions  (three  solid  lines)  are  all  repulsive  when  9d<0c1-,  which  results  a  usual  checkerboard 
CDW.  On  the  other  end,  both  Vii{aLx)  (solid  red)  and  Vii{aLX  +  aLv)  (solid  blue)  be¬ 
come  attractive  when  6d>9c2,  so  the  dominated  low-temperature  phase  is  triplet  px-wave 
pairing.  The  regime  between  two  angles  is  most  intriguing  in  monolayer  system,  where 
the  dominated  phase  is  bond  order  solid  [83].  In  a  bilayer  system,  however,  the  inter-layer 
interactions  (dashed  lines  in  Fig.  6.1)  provide  complicated  anisotropic,  even  in  the  regime 
9d  <  9ci  and  6d  >  ^c2-  As  we  show  in  the  Fig.  6.1,  the  inter-layer  interaction  is  attractive 
against  the  repulsive  intra-layer  ones,  and  it  brings  up  a  competition  between  the  inter-layer 
pairing  and  in-plane  conventional  CDW.  Similar  story  happens  on  the  other  end,  0d>9c2, 
and  question  rises  how  inter-layer  repulsion  affect  the  intra-layer  attractive  interaction.  Un¬ 
der  this  circumstances,  the  phase  boundary  can  no  long  be  roughly  determined  by  just  two 
critical  angles  [122]. 

Since  we  also  consider  the  interlayer  tunneling  (t^)  in  our  model,  the  resulting 
Hamiltonian  in  momentum  space  is  described  by  bonding  and  anti-bonding  basis.  Also, 
intra-  and  inter-band  interaction  will  be  dressed  by  both  intra-  and  inter-layer  interaction. 
For  further  simplicity,  we  mainly  focus  on  one  dipole  per  lattice  site,  saying  p  =  0.  Thus 
there  is  (tt,  tt)  nesting  between  bonding  and  anti-bonding  bands  due  to  commensurate  filling 
on  square  lattice. 
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Figure  6.2:  Phase  diagram  in  terms  of  the  dipole  orientation  {9d,4’d)  for  half-filled  single 
species  dipolar  fermions  on  a  square  lattice  bilayer  with  layer  separation  (a)  =  I.Sol, 

(b)  Iz  =  CLL,  (c)  Iz  =  0.7ai,  and  the  interlayer  tunneling  tz  =  O.lf.  Sketched  real  space 
configuration  for  (d)  CDWp  and  (e)  CDW±  with  a  vr-phase  resonance,  where  the  dashed 
red  lines  indicate  the  links  between  upper  layer  and  lower  layer,  (f)-(k)  The  corresponding 
gap  symmetry  forms  of  each  instability  are  shown  with  matching  color  and  used  data  points 
in  (b)  lz  =  aL-  The  magnitude  (not  to  scale)  is  given  by  shared  color  bar. 


6.3  Result 

6.3.1  Different  Inter-layer  separation 

We  show  our  main  result  in  Fig.  (6.2),  which  contains  phase  diagrams  obtained 
by  FRG  using  M  =  36  patches  on  each  band.  We  also  check  the  results  from  M  =  44 
patches,  which  qualitatively  agree  with  each  other.  We  first  set  layer  separation  lz  =  l.3aL- 
The  phase  diagram  is  shown  in  Fig. 6. 2(a),  and  it  is  similar  to  the  results  of  monolayer 
system  [83]  and  phases  are  still  separated  by  two  critical  angle  roughly.  For  small  On,  both 
layers  form  a  conventional  s-wave  CDW  (red  circle)  without  sign  change  in  gap  symmetry 
form,  which  shown  in  fcDWs  fo  FiS-  6-2(g).  Then,  the  Py-wave  CDW  in  Fig.  6.2(f)  is 
presence  between  two  critical  angles  (pink  circle  in  Fig.  6.2(a)).  When  is  closed  to 
45°  at  the  same  polar  angle,  the  conventional  CDW  has  a  mixed  d-wave  component,  the 
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fcDWs+d  shown  at  Fig.  6.2(h).  As  expected,  the  p-wave  triplet  pairing  (Fig.  6.2(j))  is 
dominated  when  polar  angle  is  larger  {9d  >  0c2)  and  azimuthal  angle  is  not  closed  to  45° 
(green  triangle  in  Fig.  6.2(a)).  Comparing  to  the  case  in  monolayer,  the  major  difference  is 
that  we  found  a  new  phase  near  the  regime  (0£)  >  0c2,  </>77~45°),  where  it  was  dominated  by 
p-wave  triplet  pairing.  Instead,  system  develops  a  CDW±,  black  circle  in  Fig.  6.2(a),  due  to 
the  inter-layer  repulsion,  which  is  green  dashed  line  in  Fig.  6.1(b).  It  is  worthy  to  mention 
that  the  triplet  pairing  in  monolayer  system  is  coming  from  the  strongest  attraction  along 
diagonal  direction,  which  is  solid  blue  line  Fig.  6.1(b).  With  the  presences  of  the  second 
layer,  however,  the  interlayer  repulsion  drives  the  system  to  conventional  charge  density 
wave,  where  the  diagonal  attraction  is  favor  this  state  as  well.  At  this  point,  we  can  see 
that  the  gap  symmetry  form,  fcDW±  in  Fig-  6.2(k),  carries  a  vr-phase  resonance  between 
two  bands.  That  indicates  two  conventional  CDWs  in  each  layer,  furthermore  CDW  at  A 
sub-lattice  of  one  layer  and  CDW  at  B  sub-lattice  of  the  other  layer,  and  the  real  space 
configuration  is  sketched  in  Fig.  6.2(e). 

Next,  we  discuss  the  smaller  layer  separation,  =  ul,  and  the  phase  diagram  is 
shown  in  Fig.  6.2(b).  It  is  easy  to  understand  that  smaller  layer  separation  means  inter-layer 
interaction  plays  a  more  important  role,  since  the  dipole-dipole  interaction  decays  roughly 
~  1/r^.  Under  this  set-up,  the  parameter  regime  which  was  dominated  by  conventional 
CDW  on  the  phase  diagram  shrinks.  Instead,  the  dipoles  which  aligned  perpendicular  to  the 
plane  form  a  pairing  state  (magenta  triangle),  and  it  is  in-phase  between  bands  because  the 
origin  of  the  inter-layer  attraction.  Also,  most  part  of  the  regime  is  now  dominated  by  the 
unconventional  Py-wave  CDW  (also  known  as  bond  order  solid  in  Ref.  [83]).  Interestingly, 
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the  gap  symmetry  form  in  this  CDWp  instability,  which  is  shown  in  Fig.  6.2(f)  as  fcDWp^ 
has  Py-wave  form  in  each  band  and  out-of-phase  (vr-phase  resonance)  between  bands  linked 
by  nesting  vector  (tt,  vr).  In  real  space,  we  have  the  density  modulation  on  alternating  bonds 
along  y-direction,  which  can  be  written  as  [92] 

OcDWp(7’)  =  i-^T(i{x,y)ca{x,y  +  j)  (6.4) 

a;ij=:kl 

In  Fig.  6.2(d),  we  show  the  real  space  configuration  for  this  out-of-phase  CDWp  instability. 
The  sing  structure  between  bands  indicates,  analogy  to  CDW±,  the  particle  wave- function 
distributes  on  A-B  bond  along  y-direction  on  one  layer,  and  B-A  bond  on  the  other.  Here, 
the  physical  origin  that  particles  avoid  staying  on  site  to  form  a  conventional  s-wave  CDW 
is  frustration  from  other  long-range  interactions,  since  on-site  density  modulation  is  not  able 
to  minimize  free  energy.  The  reason  that  this  exotic  density  wave  has  a  larger  parameter 
range  in  phase  diagram  is  due  to  the  bilayer  effect  with  interlayer  tunneling  and  long-range 
nature  of  dipole-dipole  interaction.  In  Fig.  6.1(a),  we  can  see  the  tilting  polar  angle  changes 
the  interaction  strength.  Although  nearest  intra-layer  repulsion  Vii{aLx)  and  inter-layer 
attraction  Fi2(0)  become  smaller  as  polar  angle  tilts  down,  it  favors  the  inter-layer  next- 
nearest  neighbor  attraction  Vi2{aLx).  By  comparing  the  Figs.  6.2(a)  and  (b)  at  6d  >  dc2, 
we  also  discover  that  in-plane  p-wave  pairing  is  not  stable  against  inter-layer  repulsion, 
so  CDW±  dominate  larger  parameter  regime  in  phase  diagram  than  the  previous  case, 
Iz  — 1.3(1^. 

Finally,  we  take  two  layers  further  closed  to  each  other  to  Iz  =  0.70^,  and  show 
the  phase  diagram  in  Fig.  6.2(c).  At  this  case,  entire  phase  diagram  is  mostly  dominated 
by  inter-layer  interaction,  where  small  polar  angle,  0d  results  in  pairing  due  to  the 
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strong  inter-layer  attraction  and  CDW±  at  large  polar  angle,  9d  >0c2-  Between  two  critical 
angles,  however,  both  CDWp  and  in-plane  p-wave  pairing  still  survives.  This  is  because 
this  parameter  regime  is  less  affected  by  the  inter-layer  interaction  ,  and  the  physics  still 
determined  by  the  intra-layer  interaction  mostly. 

6.3.2  Away  from  Half  Filling 

We  also  study  the  case  when  dipole  fermions  are  away  from  half-filling.  Since 
there  is  no  nesting  between  two  FSs,  which  is  the  most  important  feature  for  density  wave 
to  develop,  we  can  no  long  find  any  exotic  density  wave  states.  Beside  that,  due  to  the 
anisotropic  nature  of  dipole-dipole  interaction,  attractive  interaction  is  always  presences, 
which  would  drive  the  system  into  pairing  instability.  If  the  polar  angle  of  dipoles  is  smaller 
than  critical  angle  0c2,  the  inter-layer  attraction  would  play  a  key  ingredient  to  form  a 
p-wave  pairing.  On  the  other  hand,  the  intra-layer  would  be  negative  if  the  polar  angle 
is  larger  than  0c2-  Because  the  different  driving  process  in  this  two  triplet  pairing,  the 
resulting  gap  symmetry  forms  are  different  as  well.  In  Fig.  6.2(d),  we  can  see  the  deference 
between  fscs*  and  fscSp  ■  The  former  one  is  in-phase  under  flipping  orbital  index,  but  the 
later  one  is  out-phase.  The  inter-layer  attraction  trigger  the  p-wave  pairing  with  symmetry 
form  Jbcs*  )  so  the  orientation  of  this  p-wave  pairing  doesn’t  depend  on  the  azimuthal  angle 
4>d  of  dipoles.  The  later  intra-layer  attraction,  however,  is  in-plane,  so  the  resulting  pairing 
symmetry  form  will  depend  on  the  orientation  of  azimuthal  angle  (pn  of  dipoles. 
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6.4  Summary 

We  perform  the  FRG  calculation  on  single  species  dipolar  fermions  on  square 
lattice  bilayer  with  non-zero  inter-layer  tunneling.  Under  different  tilting  angles  of  dipoles, 
we  discover  different  phases,  which  are  related  to  the  nature  of  long-range  and  anisotropic 
dipole-dipole  interactions.  For  larger  layer  separation,  the  phase  diagram  is  similar  to  the 
one  in  monolayer  system,  and  roughly  described  by  two  critical  angles.  When  two  layers 
are  closed  to  each  other,  the  inter-layer  interaction  plays  a  important  role.  Due  to  the 
strong  inter-layer  attraction  at  small  polar  angle,  the  system  form  a  paring  order  instead 
of  conventional  CDW.  The  correlated  CDW±  dominated  when  polar  angle  is  large,  because 
of  the  strong  inter-layer  repulsion.  When  the  layer  separation  is  closed  to  optimal  regime, 
we  discover  the  exotic  bond  order  wave  has  a  larger  parameter  range  on  phase  diagram. 
Also  this  quantum  state  carries  a  vr- resonance  between  layers.  Our  result  shows  the  dipolar 
fermions  on  square  lattice  bilayer  can  realize  exotic  charge  density  wave,  and  observe  the 
correlation  between  layers.  The  estimated  critical  temperature  of  these  phases  is  around 
Tc  =  ^.QQTp,  which  is  still  lower  than  recently  experimental  achievement  in  dysprosium  [93], 
rss0.2Tp.  We  expect  those  exotic  quantum  phases  can  be  observed  in  near  future. 

Probing  order  parameters  with  high  angular- momentum  order  is  always  a  challenge 
topic  in  condensed  matter  experiments.  In  ultracold  atoms,  there  are  several  proposed 
methods  to  perform  such  measurements  [112,  84,  113,  115,  114].  For  example,  a  pump- 
probe  scheme  provides  a  phase  sensitive  probe  of  the  symmetry  of  order  parameter,  which 
shows  different  pattern  of  the  density-density  correlation  [115].  Moreover,  analyzing  two- 
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particle  interference  of  atom-atom  density  correlations,  people  found  the  related  phase  of 
different  components  in  FS  is  revealed  in  the  atomic  noise  correlations [114]. 
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Chapter  7 


Conclusion 


In  this  chapter,  we  summarize  the  primary  conclusions  of  the  work  we  have  done 
in  the  entire  dissertation. 

The  first  work  in  chapter  3,  the  phase  diagram  of  an  extended  Hubbard  model 
on  a  two-dimensional  square  lattice  is  shown,  and  low  energy  ground  state  depends  on-site 
U,  nearest-neighbor  V  interactions,  and  the  filling  fraction  n.  From  our  FRG  calculation, 
the  dxy-SC  emerges  in  the  regime  of  U  >  0  and  H  >  0,  which  is  in  the  proximity  of 
charge-density-wave  order.  Upon  the  destruction  of  Fermi  surface  nesting,  a  time-reversal 
breaking  (px  +  ipy)-wave  superconducting  state  arises  when  nearest-neighbor  interaction  is 
attractive  U  <  0.  We  show  that  the  {px  -|-  ipy)-SC  on  a  square  lattice  under  a  nearest- 
neighbor  attraction  is  the  generic  behavior  when  filling  fraction  is  low  which  results  in  no 
Fermi  surface  nesting.  This  {px  +  ipy)-SC  state  can  be  used  to  engineer  a  Majorana  zero 
mode  in  the  presence  of  a  magnetic  field. 
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The  second  work  in  chapter  4,  we  study  the  phase  diagram  of  a  spin-polarized 
system  under  different  polarization  (imbalance).  When  system  is  near  balanced  regime,  the 
ground  state  remains  SDW  feature.  As  the  density  imbalance  increases  between  spin-up 
and  spin-down  particles,  the  half-filled  majority  species  forms  a  stripe  density  wave  due 
to  the  long-range  screening  attractive  interaction,  instead  of  the  usual  SDW  instability. 
At  the  strong  coupling  regime,  the  unconventional  p-SC  dominates  over  the  density  wave 
instability  due  to  the  screening  long  range  intra-species  attraction.  We  observed  that  the 
competition  between  majority  and  minority  p-SC  depends  on  the  polarization  and  nesting 
of  their  Fermi  surface.  Also,  the  critical  temperature  of  the  p-SC  is  higher  than  previous 
studies  when  the  imbalanced  Fermi  gas  without  presence  of  lattice,  which  believed  to  be 
Kohn-Luttinger  instability. 

The  third  work  in  chapter  5,  we  discover  the  unconventional  dxy-CDW  in  a  quan¬ 
tum  degenerate  Fermi-Fermi  mixture.  When  one  species  is  half-filled  and  the  other  is 
slightly  away,  the  dxy-CDW  dominates  over  conventional  SDW.  We  proposed  a  four  patch 
toy  model  and  use  phase  space  argument  to  explain  the  physical  mechanism  of  formation 
of  dxy-CDW.  The  FRG  calculation  also  verifies  the  prediction  of  four  patch  toy  model  and 
gives  entire  phase  diagram  on  different  interaction  ratio  and  filling  fraction  of  both  species. 
We  conclude  that  the  density  imbalanced  between  species  mediates  a  long  range  attraction, 
which  favors  the  half-filled  species  to  form  dxy-CDW.  Furthermore,  we  discuss  the  setup 
for  this  Fermi-Fermi  mixture,  the  way  to  engineer  large  enough  interaction  ratio  to  achieve 
this  ground  state,  and  the  potential  detection  can  be  probed  if  the  critical  temperature  is 
reached. 
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The  last  work  in  chapter  6,  we  show  phase  diagrams  of  dipolar  fermions  on  square 
lattice  bilayer  under  different  layer  separations.  Our  results  show  the  dipolar  fermions  on 
square  lattice  bilayer  can  realize  exotic  charge  density  wave,  and  observe  the  correlation 
between  layers.  Apparently,  the  inter-layer  interaction  changes  the  low  energy  ground  state 
properties.  We  observe  the  competition  between  inter-layer  pairing  and  charge  density  wave 
within  layers  when  the  dipoles  are  aligned  along  z-direction.  When  the  dipoles  are  lying 
parallel  to  the  layers,  we  discover  the  s±-CDW  overcome  the  p-wave  paring  due  to  the  inter¬ 
layer  repulsive  interaction.  Although  the  estimated  critical  temperature  of  these  phases  is 
still  lower  than  recently  experimental  achievement  in  dysprosium  [93].  We  expect  those 
exotic  quantum  phases  can  be  observed  in  near  future. 

Finally,  we  establish  theories  that  explain  physical  mechanism  of  formation  of 
unconventional  density  waves  and  superfluities.  The  presence  of  square  lattice  and  the 
density  imbalanced  are  essential  keys  to  these  states.  From  several  proposed  models,  we 
have  show  that  the  long  range  attraction  in  the  system  can  favor  p-SC  and  other  higher 
order  superfluity.  This  long  range  attraction  can  also  affect  the  nearly  half-filled  species 
to  form  a  high  order  charge  density  wave.  The  density  imbalanced  has  a  screening  effect 
to  other  species  can  be  the  origin  of  the  long  range  attraction.  Despite  the  difficulty  of 
engineering  the  long  range  interaction,  our  theory  provides  experiments  other  way  to  achieve 
the  unconventional  density  wave  and  superfluity. 
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Appendix  A 

Interaction  and  its  symmetries 

A.l  Spin-Rotation-Invariant  Interaction 

The  most  general  spin-rotation-invariant  interaction  can  be  described  by  means 
of  equal  spin  (a  =  a')  quantum  numbers  term  and  another  with  opposite  spin  (a  =  —cr') 
quantum  number,  saying  C/|| (fci,  ^2,  %)  and  U±{ki,  k2, 

E  E  t/||  (fcl ,  ^2  ;  k^'j^ ^(7^4  <7 k2  ^crfcl 

cr  fcifc2fc3 

+  A2  A3)^o-fc3^-(7fc4^-(7fc2'i'o-fci  •  (A.l) 

There  are  other  ways  to  write  down  these  interaction  by  different  quantum  numbers,  for 
example,  charge-charge  and  spin-spin  interactions, 

fc2,  k3)C{k2,  k^)C{ks,  fcl)  +  Us{ki,k2,  fc3)S(fc2,  ^4)  '  8(^3,  ^i)  (A.2) 

kik2k3 
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,  where  basis  C  and  component  Si  are  given 

C{k‘i,ki)  =  ^g-fca^o-fci; 

(7 

Si{k3,ki)  =  y~]  ^o-kaO'aa/'^cT'ki 

CFcr' 

with  Pauli  matrixes  ct*. 

Also,  we  can  use  symmetry  {\a  +  a'\  =  0)  and  anti-symmetry  {\(T  +  a'\  =  \/2)  wave 
functions, 

^  s{k4,k3)Us{ki,k2,k3)s{k2,ki)  +  ^  iy{ki,kii)U A{Kl,k2,kii)ty{k2,ki)  {A.3) 

kik2k$  2^=0, dll 

,  where  these  states  are 

s{k2,ki)  =  ^ 

^  cr 

to{k2ik\)  — ^  'ho-fcj 'h— (jfci ) 

^  fj 

t±i{k2,ki)  =  • 

Before  we  start  to  figure  out  their  relations,  we  discuss  the  symmetry  properties  of  inter¬ 
actions  first.  For  these  coupling  function  we  just  mentioned,  we  can  dehne  two  exchange 
operators  as 

XT{ki,  k2,  ks,  ki)  =  P{k2,  ki,k^,  ^4)  (A. 4) 

and 

XP(ki,  k2,  k^,  k^)  =  P(k\,  k2,  ^4,  ^3)  (A.5) 

,  and  time-inversion  operator 

TP{ki,k2,ks,k4)  =  P{ks,k4,ki,k2)  .  (A. 6) 
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We  can  check  the  symmetry  from  Eq.(A.l).  First,  we  check  it  on  time-reversal 


rE  E  U (fcicr,  k2a',  fc30-)’J'afc3^<7'fc4^a'fc2^<Tfci  = 

CFCr' 

^E  E  [C/||(fci,  ^2,  fc3)'ko-fc3^o-fc4'^'(7fc2^o-fci  +  U±{ki,k2,  k^)^ -ak4^ -ak2^ aki]  • 

cr  kik2k3 


Then,  it  is  easily  to  verify 


L.H.S.  =  '^  ^  [/(fescr,  fc4cr',feiCj)TCTfci'I'<T'fc2^(T'fc4^fTfc3  = 

crcr'  fc3,/C4,fci 

E  E  [C/||(fe3,  ^4,  fcl)'kcrfci'ko-fe2'^'o-fc4'^'<Tfc3  +  U±{ks,  ki,  fcl)^crfci'k-o-fc2'^'-crfc4'^'<Tfe3] 

O'  ksk^ki 

=  R.H.S. 


Also,  these  coupling  functions  satisfy  two  symmetries,  saying 


TT  = 

T  time-inversion 

(A.7) 

XXT  = 

T  exchange  symmetry 

(A.8) 

also  XT  = 

XT  HTT  =  T  . 

(A.9) 

A.  1.1  Parallel  and  antiparallel  spin 


In  this  spin-rotation-invariant  interaction,  we  write  down  their  basis  term  by  term 
to  figure  out  their  relations.  Let’s  start  from  charge  and  spin  quantum  numbers  represen¬ 
tation  hrst.  It  is  defined  as  usual  for  charge  quantum  number. 


C(fc2,fc4)C'(fc3,fcl) 


(7<7^ 

(A.IO) 

gct' 

(A.ll) 
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,  and  for  spin  quantum  number, 

S(fc2,fc4)-S(fe3,fcl)  =  EE  ^  ^ a"k30-a"a"''^ a"'ki  (A. 12) 

i  ua'  <T"a"' 

=  Y1  'Yh  ^"ki^ aki^ a'ki'^ <T"'ki  ■  (A.13) 

i  <jG'a"a'" 

Then,  we  can  follow  above  calculation  by  Fierz  identity, 

3 

y~!(^ao-')('^g"g-"')  =  25o-'(t''<5<t(7'"  “  ^au'^a"a"'  (A.  14) 

i=l 

,  and  arrive 

8(^2,  ^4)  •  8(^3,  fcl)  =  (Tki^ a'k2^ aki  “  ^ ak^^ a'ki'^ a'k2^ uki)  ■  (A. 15) 

cfcf' 

Bring  this  two  into  Eq.(A.2),  we  get 


'Y  ^2,  %)  +  Us{ki,  k2,  ks)^ aki'^ cTk2'^ aki 

ki,k2,k3  a 

+  Y^  y~]([/c(fcl,  fc2,  fcs)  ~  Us{kl,k2,  k2,)^ak3^  —  ak^^  —  (jk2^(7ki 
ki,k2iks  cr 

-2  E  E  Us{ki,  k2,  ks)^^k3^ -ak4'^ -aki'^ak2)  ■  (A. 16) 

ki,k2,k3  cr 

The  last  term  can  be  written  down  as 

E  E  Us{k2,  ki,  fc3)T^fc3T_^fc4T_^fe2^afci  (A.17) 

ki,k2,ks  ^ 

,  so  conclusion  can  be  made  by  comparing  Eq.(A.l)  and  Eq.(A.16) 

U\\  =  Uc  +  Us  (A.  18) 

U±  =  Uc- Us-  2XUs  .  (A.  19) 

We  can  solve  this  by  setting 

Us  =  -^Ui_  (A.20) 
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,  then  we  will  find 

Uc=\{2-X)U^  .  (A.21) 

Next,  we  take  this  back  to  Eq.(A.18),  and  find 

=  .  (A.22) 

A. 1.2  Singlet-Singlet  and  Triplet-Triplet 

Second,  if  we  use  the  singlet  and  triplet  representation.  From  the  symmetry  state, 
which  has  \a  +  (t'\  =0 

s(fc4,  k3)Us{ki,k2,  ks)s{k2,  ki)  =  -  ^  Us{ki,k2,  ks)^ crk40'0-''^ a'k2'^ -a'ki 

(7g' 

=  2  X/  ^s{kl,k2,  fcs)  aki^ -aki^ -ak2^ (Tki  “  ^ ak^^ -ak^^ uk2^ -uki) 

(7 

and  the  anti-symmetry 

iuikA,  k3)UA{ki,  k2,  k3)U{k2,  ki)  = 

V 

UA{ki,  k2,  fcs)  [io{k4,  k3)to{k2,  fei)  +  t+i(fc4,  fe3)4i(^2,  ki)  -5  t-i(fe4,  k3)t-i{k2,  ki)] 

=  UA{kl,k2,  %)  J  ^-ak3^ak4'^a'k2^-a'ki  +  Y  ^<^^3  ^f7fc4  'I'afei  (A.23) 

.  a<x' 

Bring  them  back  to  Eq.(A.3),  we  apply  the  relation  on  Ug  term 
X  Y^  s{k4,k3)Us{ki,k2,k3)s{k2,ki)  =  Y^  s{k4,k3)Us{ki,k2,k3)s{k2,ki)  (A. 24) 

kik2k3  kik2k2 

as  well  as  on  Ua  term,  and  we  arrive 

Y(Us  +  UA)^ak,^  —  ak2i^—ak2^<7hi  H“  ^  ^  ^ ak^^ ak^^ ak2^ (jki 
fclfc2^3  L  cr  cr 
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This  gives  us  the  relation 

=  Us  +  Ua 

U\\  =  Ua  (A.25) 

,  also  the  symmetry  and  anti-symmetry  argument  apply 

Us  =  1{1  +  X)U± 

Ua  =  ^{1-X)U^.  (A.26) 


A. 2  Broken  SU(2)  Symmetry  Interaction 


In  order  to  analyze  the  RG  flow 

in  this  non-SU(2)  symmetry  case. 

we  need  to  go 

back  to  their  original  basis,  saying 

C{k3,k,)  = 

a 

(A.27) 

Si{k3,ki)  = 

'y  ''y  ^ crks'^acr'^ <y'ki 

CFcr' 

(A.28) 

for  charge  and  spin,  and 

s{k2,ki)  = 

(A.29) 

to{k2,ki)  = 

^  u 

(A.30) 

t±i{k2,ki)  = 

(A.31) 

for  singlet  and  triplet  channel. 
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A. 2.1  Rewrite  General  Interaction 

The  four-leg  vertices  can  be  rewritten  as  following, 

^  =  EE  (1234)V'<7'  (3)V’a(4)V’a(2)V’o-'  (1) 

1234  aa' 

=  E^E  (3)V’a(4)V’<7(2)iAa'  (1) 

1234  ^  cro-' 

+C/<,,/(2134)V;,,(3)^a(4)V’.(l)V’.'(2)]  .  (A.32) 

Let  us  define 


(1234)  =  U,,, 
t/o-o-'(2134)  =  XUacr' 


,  and  spell  all  terms  out  explicitly 

1234  ^ 

+  ^t^4t^t(3)^t(4)V’t(2)V't(l)  +  ^t^t4^4(3)^t(4)V't(2)V’t(l) 

-  ^A[/^t^t(3)^t(4)V’t(2)V^t(l)  -  ^A[/t^i/;^(3)V;t(4)iA^(2)V’t(l)  .  (A.33) 

In  order  to  use  charge-charge  and  spin-spin  separation,  we  re-write  it  into  the  following 
form 

^  =  E  -  At/^^)V;t(3)iAt(l)^t(4)V't(2)  +  Vu  -  AC/^)V;^(3)i/.^(1)V;^(4)i/.^(2) 

1234  ^ 

+  ^t^4t^t(3)V’t(l)^4(4)V't(2)  +  ^t^t4^4(3)V’t(l)^t(4)V’t(2) 

-  ^A[/^t^t(3)V^t(l)^4(4)V^t(2)  -  ^AC/tt^t(3)V’t(l)^t(4)V’t(2)  •  (A.34) 
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A. 2. 2  Charge-Charge  and  Spin-Spin 

By  using  the  definition  in  the  beginning,  we  can  express  these  fields  in  terms  of 
C(fc3,  ki)  =  C{3, 1)  and  Si{ks,  ki)  =  Si{3, 1),  saying 


^(C(3,l)  +  S,(3,l)) 

(A.35) 

^(C(3,l)-S,(3,l)) 

(A.36) 

isU3.1) 

(A.37) 

^4(3)V't(l)  = 

^S_(3.1). 

(A.38) 

Bring  these  into  Eq.(A.34),  we  have 

^  =  E  I  +  Uu  -  +  Un  +  Uit)  C{2, 4)C{3, 1) 

1234  ^ 

+  ^  (C/tt  -  XUn  +  Uu-  XUu  -  Uu  -  U^t)  ^(2, 4)5,(3, 1) 

-  ^A[/n^(2,4)5_(3,l)  -  jAC/^tM2,4)5+(3,l) 

+  J  (Ctt  -  XU^  -u^  +  xu^  +  Un  -  Un)  T,i2, 4)C(3, 1) 

+  J  {Un  -  xun  -  Uu  +  xUu  -  u^  +  u^)  c{2, 4)5. (3,  i)  (A.39) 


,  then  we  can  conclude  that 

14  =  ^{Un-xUn  +  Uu-xUu  +  Uu  +  Un)  (a.4o) 

K.  =  ^(Un-xUn  +  Un-xUn-Un-Un)  (a.4i) 

14^  =  -\xUu  (A.42) 

=  -\xUn.  (A.43) 
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A. 2. 3  Singlet-Singlet  and  Triplet-Triplet 

Once  again,  we  are  going  to  use  the  definition  in  beginning  of  this  chapter  and  use 
them  to  represent  these  fields,  saying 

•5(2, 1)  =  ^  [V^t(2)V’t(l)  -  V't(2)V’t(l)] 

^o(2, 1)  =  ^  [V^t(2)V’^(l)  +  V'^(2)V’t(l)] 

4(2,1)  =  4(2)4(1) 

t_(2,l)  =  4(2)4(1)  (A.44) 

and 

s(4,3)  =  ^  [4(3)4(4) -4(3)4(4)] 

io(4,3)  =  ^  [4(3)4(4) +4(3)4(4)] 

4(4,3)  =  4(3)4(4) 

4(4,3)  =  4(3)4(4)-  (A.45) 

Therefore,  the  interaction  will  be  expressed  as 

4"  =  E  -  A4t)4(4,  3)4(2, 1)  +  \{U^  -  X4^)t-(4,  3)t_(2, 1) 

1234  ^ 

+  4t  +  At/tl  +  A4t)^(4,  3)s(2, 1) 

^(44  -  +  At/tl  -  A4t)^o(4,  3)to(2, 1) 

^(44  -  +  A44  -  A4t)io(4,  3)s(2, 1) 

^(44  -  -  A44  +  A4t)44, 3)to(2, 1)  •  (A.46) 
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Following  by  these,  we  can  define  that 


^BCS  — 

\{Un  +  Uy  +  xUy  +  xUy) 

(A.47) 

T/iO  — 

^BCS  — 

\{un-uy  +  xuy-xuy) 

(A.48) 

yt+  _ 

^BCS  ~ 

-{Uy  -  XUy) 

(A.49) 

^BCS  — 

]^{Uy-XUy)  . 

(A.50) 
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